2023-02-07 16:53:58,871 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-07 16:53:59,084 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa 2023-02-07 16:53:59,098 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-07 16:53:59,098 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603, deleteOnExit=true 2023-02-07 16:53:59,099 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-07 16:53:59,099 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/test.cache.data in system properties and HBase conf 2023-02-07 16:53:59,100 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/hadoop.tmp.dir in system properties and HBase conf 2023-02-07 16:53:59,100 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/hadoop.log.dir in system properties and HBase conf 2023-02-07 16:53:59,101 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-07 16:53:59,101 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-07 16:53:59,101 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-07 16:53:59,204 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-07 16:53:59,519 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-07 16:53:59,524 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-07 16:53:59,524 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-07 16:53:59,524 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-07 16:53:59,525 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-07 16:53:59,525 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-07 16:53:59,525 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-07 16:53:59,526 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-07 16:53:59,526 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-07 16:53:59,526 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-07 16:53:59,527 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/nfs.dump.dir in system properties and HBase conf 2023-02-07 16:53:59,527 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/java.io.tmpdir in system properties and HBase conf 2023-02-07 16:53:59,528 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-07 16:53:59,528 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-07 16:53:59,528 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-07 16:53:59,983 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-07 16:53:59,987 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-07 16:54:00,595 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-07 16:54:00,750 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-07 16:54:00,765 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 16:54:00,801 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 16:54:00,829 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/java.io.tmpdir/Jetty_localhost_localdomain_36599_hdfs____.rwb1hw/webapp 2023-02-07 16:54:00,979 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:36599 2023-02-07 16:54:00,988 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-07 16:54:00,988 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-07 16:54:01,576 WARN [Listener at localhost.localdomain/43315] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 16:54:01,690 WARN [Listener at localhost.localdomain/43315] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-07 16:54:01,708 WARN [Listener at localhost.localdomain/43315] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 16:54:01,715 INFO [Listener at localhost.localdomain/43315] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 16:54:01,720 INFO [Listener at localhost.localdomain/43315] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/java.io.tmpdir/Jetty_localhost_36263_datanode____.3q9a7h/webapp 2023-02-07 16:54:01,796 INFO [Listener at localhost.localdomain/43315] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36263 2023-02-07 16:54:02,085 WARN [Listener at localhost.localdomain/44409] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 16:54:02,095 WARN [Listener at localhost.localdomain/44409] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-07 16:54:02,099 WARN [Listener at localhost.localdomain/44409] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 16:54:02,101 INFO [Listener at localhost.localdomain/44409] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 16:54:02,109 INFO [Listener at localhost.localdomain/44409] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/java.io.tmpdir/Jetty_localhost_43785_datanode____5jkzg/webapp 2023-02-07 16:54:02,190 INFO [Listener at localhost.localdomain/44409] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43785 2023-02-07 16:54:02,198 WARN [Listener at localhost.localdomain/33395] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 16:54:02,217 WARN [Listener at localhost.localdomain/33395] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-07 16:54:02,221 WARN [Listener at localhost.localdomain/33395] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-07 16:54:02,224 INFO [Listener at localhost.localdomain/33395] log.Slf4jLog(67): jetty-6.1.26 2023-02-07 16:54:02,231 INFO [Listener at localhost.localdomain/33395] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/java.io.tmpdir/Jetty_localhost_39321_datanode____.w0c1al/webapp 2023-02-07 16:54:02,327 INFO [Listener at localhost.localdomain/33395] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39321 2023-02-07 16:54:02,337 WARN [Listener at localhost.localdomain/34435] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-07 16:54:04,066 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x829a35041a5e9c3c: Processing first storage report for DS-e1f03cc1-4f41-4594-abe0-feefe6a13525 from datanode 74115c84-26e5-4d15-9a8f-50c13dd5b9f8 2023-02-07 16:54:04,068 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x829a35041a5e9c3c: from storage DS-e1f03cc1-4f41-4594-abe0-feefe6a13525 node DatanodeRegistration(127.0.0.1:43233, datanodeUuid=74115c84-26e5-4d15-9a8f-50c13dd5b9f8, infoPort=36407, infoSecurePort=0, ipcPort=33395, storageInfo=lv=-57;cid=testClusterID;nsid=937079442;c=1675788840051), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-07 16:54:04,068 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa4bae20b93fb4d7a: Processing first storage report for DS-84b85dd0-655b-460e-925e-a1c8720d74ef from datanode a02ccce2-1098-451e-aa52-03b9bc2cf94f 2023-02-07 16:54:04,068 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa4bae20b93fb4d7a: from storage DS-84b85dd0-655b-460e-925e-a1c8720d74ef node DatanodeRegistration(127.0.0.1:41327, datanodeUuid=a02ccce2-1098-451e-aa52-03b9bc2cf94f, infoPort=45179, infoSecurePort=0, ipcPort=44409, storageInfo=lv=-57;cid=testClusterID;nsid=937079442;c=1675788840051), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 16:54:04,068 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x95399d26a62b2e44: Processing first storage report for DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9 from datanode 3f3daf44-903e-4489-b9da-57bdf6b56058 2023-02-07 16:54:04,068 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x95399d26a62b2e44: from storage DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9 node DatanodeRegistration(127.0.0.1:36649, datanodeUuid=3f3daf44-903e-4489-b9da-57bdf6b56058, infoPort=38703, infoSecurePort=0, ipcPort=34435, storageInfo=lv=-57;cid=testClusterID;nsid=937079442;c=1675788840051), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 16:54:04,068 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x829a35041a5e9c3c: Processing first storage report for DS-731092a4-6d54-421c-b6d7-6c17625cfd0f from datanode 74115c84-26e5-4d15-9a8f-50c13dd5b9f8 2023-02-07 16:54:04,069 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x829a35041a5e9c3c: from storage DS-731092a4-6d54-421c-b6d7-6c17625cfd0f node DatanodeRegistration(127.0.0.1:43233, datanodeUuid=74115c84-26e5-4d15-9a8f-50c13dd5b9f8, infoPort=36407, infoSecurePort=0, ipcPort=33395, storageInfo=lv=-57;cid=testClusterID;nsid=937079442;c=1675788840051), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 16:54:04,069 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa4bae20b93fb4d7a: Processing first storage report for DS-4cf2d909-c675-41b6-acc8-fede4f0f25be from datanode a02ccce2-1098-451e-aa52-03b9bc2cf94f 2023-02-07 16:54:04,069 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa4bae20b93fb4d7a: from storage DS-4cf2d909-c675-41b6-acc8-fede4f0f25be node DatanodeRegistration(127.0.0.1:41327, datanodeUuid=a02ccce2-1098-451e-aa52-03b9bc2cf94f, infoPort=45179, infoSecurePort=0, ipcPort=44409, storageInfo=lv=-57;cid=testClusterID;nsid=937079442;c=1675788840051), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-07 16:54:04,069 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x95399d26a62b2e44: Processing first storage report for DS-eb980563-96b8-4a0b-b03a-1a0546f0c883 from datanode 3f3daf44-903e-4489-b9da-57bdf6b56058 2023-02-07 16:54:04,069 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x95399d26a62b2e44: from storage DS-eb980563-96b8-4a0b-b03a-1a0546f0c883 node DatanodeRegistration(127.0.0.1:36649, datanodeUuid=3f3daf44-903e-4489-b9da-57bdf6b56058, infoPort=38703, infoSecurePort=0, ipcPort=34435, storageInfo=lv=-57;cid=testClusterID;nsid=937079442;c=1675788840051), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-07 16:54:04,098 DEBUG [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa 2023-02-07 16:54:04,153 INFO [Listener at localhost.localdomain/34435] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/zookeeper_0, clientPort=50517, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-07 16:54:04,166 INFO [Listener at localhost.localdomain/34435] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50517 2023-02-07 16:54:04,173 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:04,175 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:04,818 INFO [Listener at localhost.localdomain/34435] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af with version=8 2023-02-07 16:54:04,818 INFO [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/hbase-staging 2023-02-07 16:54:05,129 INFO [Listener at localhost.localdomain/34435] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-07 16:54:05,524 INFO [Listener at localhost.localdomain/34435] client.ConnectionUtils(127): master/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 16:54:05,550 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:05,551 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:05,551 INFO [Listener at localhost.localdomain/34435] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 16:54:05,551 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:05,551 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 16:54:05,673 INFO [Listener at localhost.localdomain/34435] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 16:54:05,737 DEBUG [Listener at localhost.localdomain/34435] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-07 16:54:05,817 INFO [Listener at localhost.localdomain/34435] ipc.NettyRpcServer(120): Bind to /136.243.104.168:39383 2023-02-07 16:54:05,826 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:05,828 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:05,847 INFO [Listener at localhost.localdomain/34435] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39383 connecting to ZooKeeper ensemble=127.0.0.1:50517 2023-02-07 16:54:06,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:393830x0, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 16:54:06,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39383-0x1013e5847d70000 connected 2023-02-07 16:54:06,225 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 16:54:06,226 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:54:06,229 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 16:54:06,236 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39383 2023-02-07 16:54:06,237 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39383 2023-02-07 16:54:06,237 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39383 2023-02-07 16:54:06,237 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39383 2023-02-07 16:54:06,238 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39383 2023-02-07 16:54:06,243 INFO [Listener at localhost.localdomain/34435] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af, hbase.cluster.distributed=false 2023-02-07 16:54:06,301 INFO [Listener at localhost.localdomain/34435] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 16:54:06,301 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,301 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,301 INFO [Listener at localhost.localdomain/34435] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 16:54:06,301 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,301 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 16:54:06,305 INFO [Listener at localhost.localdomain/34435] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 16:54:06,308 INFO [Listener at localhost.localdomain/34435] ipc.NettyRpcServer(120): Bind to /136.243.104.168:43011 2023-02-07 16:54:06,310 INFO [Listener at localhost.localdomain/34435] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-07 16:54:06,315 DEBUG [Listener at localhost.localdomain/34435] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-07 16:54:06,317 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,318 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,320 INFO [Listener at localhost.localdomain/34435] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43011 connecting to ZooKeeper ensemble=127.0.0.1:50517 2023-02-07 16:54:06,329 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:430110x0, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 16:54:06,330 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43011-0x1013e5847d70001 connected 2023-02-07 16:54:06,330 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 16:54:06,332 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:54:06,333 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 16:54:06,334 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43011 2023-02-07 16:54:06,335 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43011 2023-02-07 16:54:06,336 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43011 2023-02-07 16:54:06,336 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43011 2023-02-07 16:54:06,336 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43011 2023-02-07 16:54:06,347 INFO [Listener at localhost.localdomain/34435] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 16:54:06,348 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,348 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,348 INFO [Listener at localhost.localdomain/34435] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 16:54:06,349 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,349 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 16:54:06,349 INFO [Listener at localhost.localdomain/34435] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 16:54:06,354 INFO [Listener at localhost.localdomain/34435] ipc.NettyRpcServer(120): Bind to /136.243.104.168:41937 2023-02-07 16:54:06,354 INFO [Listener at localhost.localdomain/34435] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-07 16:54:06,356 DEBUG [Listener at localhost.localdomain/34435] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-07 16:54:06,357 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,359 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,363 INFO [Listener at localhost.localdomain/34435] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41937 connecting to ZooKeeper ensemble=127.0.0.1:50517 2023-02-07 16:54:06,371 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:419370x0, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 16:54:06,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41937-0x1013e5847d70002 connected 2023-02-07 16:54:06,372 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 16:54:06,373 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:54:06,374 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 16:54:06,377 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41937 2023-02-07 16:54:06,377 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41937 2023-02-07 16:54:06,377 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41937 2023-02-07 16:54:06,378 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41937 2023-02-07 16:54:06,378 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41937 2023-02-07 16:54:06,388 INFO [Listener at localhost.localdomain/34435] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-07 16:54:06,388 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,389 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,389 INFO [Listener at localhost.localdomain/34435] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-07 16:54:06,389 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-07 16:54:06,389 INFO [Listener at localhost.localdomain/34435] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-07 16:54:06,389 INFO [Listener at localhost.localdomain/34435] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-07 16:54:06,390 INFO [Listener at localhost.localdomain/34435] ipc.NettyRpcServer(120): Bind to /136.243.104.168:40593 2023-02-07 16:54:06,391 INFO [Listener at localhost.localdomain/34435] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-07 16:54:06,392 DEBUG [Listener at localhost.localdomain/34435] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-07 16:54:06,393 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,394 INFO [Listener at localhost.localdomain/34435] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,395 INFO [Listener at localhost.localdomain/34435] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40593 connecting to ZooKeeper ensemble=127.0.0.1:50517 2023-02-07 16:54:06,482 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:405930x0, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-07 16:54:06,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40593-0x1013e5847d70003 connected 2023-02-07 16:54:06,484 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 16:54:06,485 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:54:06,486 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-07 16:54:06,486 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40593 2023-02-07 16:54:06,487 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40593 2023-02-07 16:54:06,487 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40593 2023-02-07 16:54:06,487 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40593 2023-02-07 16:54:06,488 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40593 2023-02-07 16:54:06,493 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:06,529 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-07 16:54:06,531 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:06,549 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 16:54:06,549 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 16:54:06,549 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 16:54:06,549 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-07 16:54:06,551 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:06,551 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-07 16:54:06,552 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase12.apache.org,39383,1675788844945 from backup master directory 2023-02-07 16:54:06,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-07 16:54:06,560 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:06,560 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-07 16:54:06,561 WARN [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 16:54:06,561 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:06,563 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-07 16:54:06,565 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-07 16:54:06,656 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/hbase.id with ID: 7eb5a454-6c9f-4572-9231-7a9dba40b478 2023-02-07 16:54:06,691 INFO [master/jenkins-hbase12:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-07 16:54:06,718 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:06,765 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x6d7358eb to 127.0.0.1:50517 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 16:54:06,806 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53801ebe, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 16:54:06,827 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-07 16:54:06,828 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-07 16:54:06,843 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-07 16:54:06,843 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-07 16:54:06,845 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:54:06,848 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:54:06,849 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 16:54:06,879 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store-tmp 2023-02-07 16:54:06,918 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:06,918 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-07 16:54:06,918 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:54:06,919 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:54:06,919 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-07 16:54:06,919 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:54:06,919 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:54:06,919 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675788846918Disabling compacts and flushes for region at 1675788846918Disabling writes for close at 1675788846919 (+1 ms)Writing region close event to WAL at 1675788846919Closed at 1675788846919 2023-02-07 16:54:06,921 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/WALs/jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:06,942 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C39383%2C1675788844945, suffix=, logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/WALs/jenkins-hbase12.apache.org,39383,1675788844945, archiveDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/oldWALs, maxLogs=10 2023-02-07 16:54:06,997 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK] 2023-02-07 16:54:06,997 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK] 2023-02-07 16:54:06,997 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK] 2023-02-07 16:54:07,007 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:54:07,074 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/WALs/jenkins-hbase12.apache.org,39383,1675788844945/jenkins-hbase12.apache.org%2C39383%2C1675788844945.1675788846952 2023-02-07 16:54:07,074 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK], DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK], DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK]] 2023-02-07 16:54:07,075 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:07,075 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:07,078 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-07 16:54:07,079 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-07 16:54:07,130 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-07 16:54:07,137 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-07 16:54:07,163 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-07 16:54:07,174 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:07,180 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-07 16:54:07,182 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-07 16:54:07,196 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-07 16:54:07,200 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:07,201 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63223238, jitterRate=-0.05790033936500549}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 16:54:07,201 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675788847101Initializing all the Stores at 1675788847103 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788847104 (+1 ms)Cleaning up temporary data from old regions at 1675788847188 (+84 ms)Cleaning up detritus from prior splits at 1675788847188Region opened successfully at 1675788847201 (+13 ms) 2023-02-07 16:54:07,203 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-07 16:54:07,221 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:07,221 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:07,223 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:07,225 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:54:07,252 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 26 msec 2023-02-07 16:54:07,252 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-07 16:54:07,273 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-07 16:54:07,279 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-07 16:54:07,299 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-07 16:54:07,302 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-07 16:54:07,304 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-07 16:54:07,308 INFO [master/jenkins-hbase12:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-07 16:54:07,312 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-07 16:54:07,360 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:07,362 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-07 16:54:07,362 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-07 16:54:07,378 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-07 16:54:07,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 16:54:07,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 16:54:07,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 16:54:07,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-07 16:54:07,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:07,392 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase12.apache.org,39383,1675788844945, sessionid=0x1013e5847d70000, setting cluster-up flag (Was=false) 2023-02-07 16:54:07,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:07,455 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-07 16:54:07,456 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:07,476 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:07,507 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-07 16:54:07,509 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:07,512 WARN [master/jenkins-hbase12:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.hbase-snapshot/.tmp 2023-02-07 16:54:07,591 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(952): ClusterId : 7eb5a454-6c9f-4572-9231-7a9dba40b478 2023-02-07 16:54:07,591 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(952): ClusterId : 7eb5a454-6c9f-4572-9231-7a9dba40b478 2023-02-07 16:54:07,592 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(952): ClusterId : 7eb5a454-6c9f-4572-9231-7a9dba40b478 2023-02-07 16:54:07,597 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-07 16:54:07,597 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-07 16:54:07,597 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-07 16:54:07,617 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:07,625 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-07 16:54:07,625 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-07 16:54:07,625 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-07 16:54:07,625 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-07 16:54:07,625 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-07 16:54:07,625 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase12:0, corePoolSize=10, maxPoolSize=10 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,628 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 16:54:07,629 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,629 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788877629 2023-02-07 16:54:07,631 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-07 16:54:07,635 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:07,635 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-07 16:54:07,639 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-07 16:54:07,645 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-07 16:54:07,645 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-07 16:54:07,646 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-07 16:54:07,647 DEBUG [RS:0;jenkins-hbase12:43011] zookeeper.ReadOnlyZKClient(139): Connect 0x560f95b6 to 127.0.0.1:50517 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 16:54:07,649 DEBUG [RS:2;jenkins-hbase12:40593] zookeeper.ReadOnlyZKClient(139): Connect 0x6d0d0544 to 127.0.0.1:50517 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 16:54:07,650 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-07 16:54:07,652 DEBUG [RS:1;jenkins-hbase12:41937] zookeeper.ReadOnlyZKClient(139): Connect 0x30d81752 to 127.0.0.1:50517 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 16:54:07,655 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-07 16:54:07,685 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-07 16:54:07,686 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-07 16:54:07,686 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-07 16:54:07,688 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,690 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-07 16:54:07,692 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-07 16:54:07,693 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-07 16:54:07,705 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-07 16:54:07,706 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-07 16:54:07,709 DEBUG [RS:2;jenkins-hbase12:40593] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@63cae695, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 16:54:07,709 DEBUG [RS:1;jenkins-hbase12:41937] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e1fef39, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 16:54:07,710 DEBUG [RS:2;jenkins-hbase12:40593] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67f56e34, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 16:54:07,710 DEBUG [RS:1;jenkins-hbase12:41937] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a207c22, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 16:54:07,710 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675788847709,5,FailOnTimeoutGroup] 2023-02-07 16:54:07,711 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675788847710,5,FailOnTimeoutGroup] 2023-02-07 16:54:07,711 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,711 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-07 16:54:07,712 DEBUG [RS:0;jenkins-hbase12:43011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58c86107, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 16:54:07,713 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,713 DEBUG [RS:0;jenkins-hbase12:43011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5cf7fec6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 16:54:07,713 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,716 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:07,717 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:07,717 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af 2023-02-07 16:54:07,736 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase12:43011 2023-02-07 16:54:07,736 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase12:40593 2023-02-07 16:54:07,737 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase12:41937 2023-02-07 16:54:07,753 INFO [RS:1;jenkins-hbase12:41937] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-07 16:54:07,753 INFO [RS:2;jenkins-hbase12:40593] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-07 16:54:07,754 INFO [RS:2;jenkins-hbase12:40593] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-07 16:54:07,753 INFO [RS:0;jenkins-hbase12:43011] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-07 16:54:07,754 INFO [RS:0;jenkins-hbase12:43011] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-07 16:54:07,754 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1023): About to register with Master. 2023-02-07 16:54:07,754 INFO [RS:1;jenkins-hbase12:41937] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-07 16:54:07,754 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1023): About to register with Master. 2023-02-07 16:54:07,754 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1023): About to register with Master. 2023-02-07 16:54:07,754 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:07,758 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,39383,1675788844945 with isa=jenkins-hbase12.apache.org/136.243.104.168:43011, startcode=1675788846300 2023-02-07 16:54:07,758 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-07 16:54:07,758 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,39383,1675788844945 with isa=jenkins-hbase12.apache.org/136.243.104.168:40593, startcode=1675788846388 2023-02-07 16:54:07,758 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,39383,1675788844945 with isa=jenkins-hbase12.apache.org/136.243.104.168:41937, startcode=1675788846347 2023-02-07 16:54:07,761 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/info 2023-02-07 16:54:07,762 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-07 16:54:07,766 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:07,766 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-07 16:54:07,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/rep_barrier 2023-02-07 16:54:07,770 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-07 16:54:07,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:07,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-07 16:54:07,774 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/table 2023-02-07 16:54:07,775 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-07 16:54:07,776 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:07,778 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740 2023-02-07 16:54:07,779 DEBUG [RS:1;jenkins-hbase12:41937] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-07 16:54:07,779 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740 2023-02-07 16:54:07,779 DEBUG [RS:2;jenkins-hbase12:40593] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-07 16:54:07,779 DEBUG [RS:0;jenkins-hbase12:43011] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-07 16:54:07,784 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-07 16:54:07,786 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-07 16:54:07,791 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:07,792 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63580200, jitterRate=-0.052581191062927246}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-07 16:54:07,792 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675788847755Initializing all the Stores at 1675788847757 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675788847757Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788847757Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675788847757Cleaning up temporary data from old regions at 1675788847782 (+25 ms)Cleaning up detritus from prior splits at 1675788847782Region opened successfully at 1675788847792 (+10 ms) 2023-02-07 16:54:07,792 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-07 16:54:07,792 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-07 16:54:07,792 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-07 16:54:07,792 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-07 16:54:07,792 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-07 16:54:07,794 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-07 16:54:07,795 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675788847792Disabling compacts and flushes for region at 1675788847792Disabling writes for close at 1675788847792Writing region close event to WAL at 1675788847794 (+2 ms)Closed at 1675788847794 2023-02-07 16:54:07,822 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:07,822 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-07 16:54:07,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-07 16:54:07,842 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:45983, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-07 16:54:07,842 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:53627, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-07 16:54:07,842 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:47085, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-07 16:54:07,848 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-07 16:54:07,850 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-07 16:54:07,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39383] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:07,856 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:07,856 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39383] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:07,870 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af 2023-02-07 16:54:07,870 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af 2023-02-07 16:54:07,870 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af 2023-02-07 16:54:07,870 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43315 2023-02-07 16:54:07,870 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43315 2023-02-07 16:54:07,870 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-07 16:54:07,870 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:43315 2023-02-07 16:54:07,870 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-07 16:54:07,871 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-07 16:54:07,911 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 16:54:07,911 DEBUG [RS:1;jenkins-hbase12:41937] zookeeper.ZKUtil(162): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:07,911 DEBUG [RS:2;jenkins-hbase12:40593] zookeeper.ZKUtil(162): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:07,911 WARN [RS:1;jenkins-hbase12:41937] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 16:54:07,911 DEBUG [RS:0;jenkins-hbase12:43011] zookeeper.ZKUtil(162): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:07,911 WARN [RS:2;jenkins-hbase12:40593] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 16:54:07,913 WARN [RS:0;jenkins-hbase12:43011] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-07 16:54:07,913 INFO [RS:1;jenkins-hbase12:41937] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 16:54:07,913 INFO [RS:0;jenkins-hbase12:43011] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 16:54:07,913 INFO [RS:2;jenkins-hbase12:40593] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 16:54:07,913 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:07,913 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:07,913 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:07,914 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,41937,1675788846347] 2023-02-07 16:54:07,914 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,43011,1675788846300] 2023-02-07 16:54:07,914 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,40593,1675788846388] 2023-02-07 16:54:07,925 DEBUG [RS:1;jenkins-hbase12:41937] zookeeper.ZKUtil(162): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:07,925 DEBUG [RS:2;jenkins-hbase12:40593] zookeeper.ZKUtil(162): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:07,925 DEBUG [RS:0;jenkins-hbase12:43011] zookeeper.ZKUtil(162): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:07,926 DEBUG [RS:1;jenkins-hbase12:41937] zookeeper.ZKUtil(162): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:07,926 DEBUG [RS:2;jenkins-hbase12:40593] zookeeper.ZKUtil(162): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:07,926 DEBUG [RS:0;jenkins-hbase12:43011] zookeeper.ZKUtil(162): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:07,926 DEBUG [RS:1;jenkins-hbase12:41937] zookeeper.ZKUtil(162): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:07,926 DEBUG [RS:2;jenkins-hbase12:40593] zookeeper.ZKUtil(162): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:07,926 DEBUG [RS:0;jenkins-hbase12:43011] zookeeper.ZKUtil(162): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:07,935 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-07 16:54:07,935 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-07 16:54:07,935 DEBUG [RS:1;jenkins-hbase12:41937] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-07 16:54:07,944 INFO [RS:0;jenkins-hbase12:43011] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-07 16:54:07,944 INFO [RS:1;jenkins-hbase12:41937] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-07 16:54:07,944 INFO [RS:2;jenkins-hbase12:40593] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-07 16:54:07,966 INFO [RS:0;jenkins-hbase12:43011] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-07 16:54:07,966 INFO [RS:2;jenkins-hbase12:40593] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-07 16:54:07,966 INFO [RS:1;jenkins-hbase12:41937] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-07 16:54:07,969 INFO [RS:0;jenkins-hbase12:43011] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-07 16:54:07,969 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,970 INFO [RS:2;jenkins-hbase12:40593] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-07 16:54:07,970 INFO [RS:1;jenkins-hbase12:41937] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-07 16:54:07,971 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,971 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,971 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-07 16:54:07,971 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-07 16:54:07,972 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-07 16:54:07,980 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,980 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,980 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,980 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,980 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,981 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 16:54:07,982 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 16:54:07,981 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-07 16:54:07,982 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,982 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,983 DEBUG [RS:2;jenkins-hbase12:40593] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,983 DEBUG [RS:0;jenkins-hbase12:43011] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,983 DEBUG [RS:1;jenkins-hbase12:41937] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-07 16:54:07,984 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,984 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,984 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,984 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,984 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,984 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,987 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,988 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:07,988 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,001 INFO [RS:1;jenkins-hbase12:41937] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-07 16:54:08,002 INFO [RS:2;jenkins-hbase12:40593] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-07 16:54:08,001 INFO [RS:0;jenkins-hbase12:43011] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-07 16:54:08,003 DEBUG [jenkins-hbase12:39383] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-07 16:54:08,005 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,40593,1675788846388-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,005 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,41937,1675788846347-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,005 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,43011,1675788846300-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,008 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:08,016 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:08,016 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:08,016 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:08,016 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:08,020 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,40593,1675788846388, state=OPENING 2023-02-07 16:54:08,028 INFO [RS:2;jenkins-hbase12:40593] regionserver.Replication(203): jenkins-hbase12.apache.org,40593,1675788846388 started 2023-02-07 16:54:08,028 INFO [RS:1;jenkins-hbase12:41937] regionserver.Replication(203): jenkins-hbase12.apache.org,41937,1675788846347 started 2023-02-07 16:54:08,028 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,40593,1675788846388, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:40593, sessionid=0x1013e5847d70003 2023-02-07 16:54:08,028 INFO [RS:0;jenkins-hbase12:43011] regionserver.Replication(203): jenkins-hbase12.apache.org,43011,1675788846300 started 2023-02-07 16:54:08,029 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,41937,1675788846347, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:41937, sessionid=0x1013e5847d70002 2023-02-07 16:54:08,029 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,43011,1675788846300, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:43011, sessionid=0x1013e5847d70001 2023-02-07 16:54:08,029 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-07 16:54:08,029 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-07 16:54:08,029 DEBUG [RS:1;jenkins-hbase12:41937] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:08,029 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-07 16:54:08,030 DEBUG [RS:1;jenkins-hbase12:41937] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,41937,1675788846347' 2023-02-07 16:54:08,030 DEBUG [RS:2;jenkins-hbase12:40593] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:08,029 DEBUG [RS:0;jenkins-hbase12:43011] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:08,030 DEBUG [RS:1;jenkins-hbase12:41937] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:08,030 DEBUG [RS:0;jenkins-hbase12:43011] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,43011,1675788846300' 2023-02-07 16:54:08,030 DEBUG [RS:0;jenkins-hbase12:43011] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:08,030 DEBUG [RS:2;jenkins-hbase12:40593] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,40593,1675788846388' 2023-02-07 16:54:08,030 DEBUG [RS:2;jenkins-hbase12:40593] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:08,044 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-07 16:54:08,044 DEBUG [RS:2;jenkins-hbase12:40593] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:08,044 DEBUG [RS:0;jenkins-hbase12:43011] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:08,044 DEBUG [RS:1;jenkins-hbase12:41937] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:08,054 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:08,055 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-07 16:54:08,055 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-07 16:54:08,055 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-07 16:54:08,055 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-07 16:54:08,055 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-07 16:54:08,055 DEBUG [RS:1;jenkins-hbase12:41937] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:08,055 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-07 16:54:08,056 DEBUG [RS:1;jenkins-hbase12:41937] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,41937,1675788846347' 2023-02-07 16:54:08,057 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-07 16:54:08,055 DEBUG [RS:2;jenkins-hbase12:40593] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:08,057 DEBUG [RS:2;jenkins-hbase12:40593] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,40593,1675788846388' 2023-02-07 16:54:08,057 DEBUG [RS:2;jenkins-hbase12:40593] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-07 16:54:08,057 DEBUG [RS:1;jenkins-hbase12:41937] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-07 16:54:08,056 DEBUG [RS:0;jenkins-hbase12:43011] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:08,057 DEBUG [RS:0;jenkins-hbase12:43011] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,43011,1675788846300' 2023-02-07 16:54:08,057 DEBUG [RS:0;jenkins-hbase12:43011] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-07 16:54:08,057 DEBUG [RS:2;jenkins-hbase12:40593] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-07 16:54:08,058 DEBUG [RS:0;jenkins-hbase12:43011] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-07 16:54:08,058 DEBUG [RS:2;jenkins-hbase12:40593] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-07 16:54:08,058 INFO [RS:2;jenkins-hbase12:40593] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-07 16:54:08,058 INFO [RS:2;jenkins-hbase12:40593] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-07 16:54:08,058 DEBUG [RS:0;jenkins-hbase12:43011] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-07 16:54:08,058 INFO [RS:0;jenkins-hbase12:43011] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-07 16:54:08,058 INFO [RS:0;jenkins-hbase12:43011] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-07 16:54:08,059 DEBUG [RS:1;jenkins-hbase12:41937] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-07 16:54:08,059 DEBUG [RS:1;jenkins-hbase12:41937] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-07 16:54:08,059 INFO [RS:1;jenkins-hbase12:41937] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-07 16:54:08,059 INFO [RS:1;jenkins-hbase12:41937] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-07 16:54:08,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:08,168 INFO [RS:0;jenkins-hbase12:43011] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C43011%2C1675788846300, suffix=, logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,43011,1675788846300, archiveDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs, maxLogs=32 2023-02-07 16:54:08,168 INFO [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C40593%2C1675788846388, suffix=, logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,40593,1675788846388, archiveDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs, maxLogs=32 2023-02-07 16:54:08,168 INFO [RS:1;jenkins-hbase12:41937] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C41937%2C1675788846347, suffix=, logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,41937,1675788846347, archiveDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs, maxLogs=32 2023-02-07 16:54:08,193 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK] 2023-02-07 16:54:08,194 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK] 2023-02-07 16:54:08,193 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK] 2023-02-07 16:54:08,195 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK] 2023-02-07 16:54:08,195 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK] 2023-02-07 16:54:08,195 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK] 2023-02-07 16:54:08,205 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK] 2023-02-07 16:54:08,205 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK] 2023-02-07 16:54:08,205 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK] 2023-02-07 16:54:08,214 INFO [RS:0;jenkins-hbase12:43011] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,43011,1675788846300/jenkins-hbase12.apache.org%2C43011%2C1675788846300.1675788848171 2023-02-07 16:54:08,214 INFO [RS:1;jenkins-hbase12:41937] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,41937,1675788846347/jenkins-hbase12.apache.org%2C41937%2C1675788846347.1675788848171 2023-02-07 16:54:08,214 DEBUG [RS:0;jenkins-hbase12:43011] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK], DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK], DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK]] 2023-02-07 16:54:08,214 DEBUG [RS:1;jenkins-hbase12:41937] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK], DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK], DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK]] 2023-02-07 16:54:08,218 INFO [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,40593,1675788846388/jenkins-hbase12.apache.org%2C40593%2C1675788846388.1675788848171 2023-02-07 16:54:08,220 DEBUG [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK], DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK], DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK]] 2023-02-07 16:54:08,246 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:08,248 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 16:54:08,251 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:36938, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 16:54:08,265 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-07 16:54:08,266 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-07 16:54:08,270 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C40593%2C1675788846388.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,40593,1675788846388, archiveDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs, maxLogs=32 2023-02-07 16:54:08,289 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK] 2023-02-07 16:54:08,290 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK] 2023-02-07 16:54:08,292 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK] 2023-02-07 16:54:08,299 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/WALs/jenkins-hbase12.apache.org,40593,1675788846388/jenkins-hbase12.apache.org%2C40593%2C1675788846388.meta.1675788848271.meta 2023-02-07 16:54:08,299 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41327,DS-84b85dd0-655b-460e-925e-a1c8720d74ef,DISK], DatanodeInfoWithStorage[127.0.0.1:43233,DS-e1f03cc1-4f41-4594-abe0-feefe6a13525,DISK], DatanodeInfoWithStorage[127.0.0.1:36649,DS-d1dad32e-f5bd-4d6e-84f9-632f9ae494a9,DISK]] 2023-02-07 16:54:08,299 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:08,303 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:08,304 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-07 16:54:08,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-07 16:54:08,320 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-07 16:54:08,323 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-07 16:54:08,324 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:08,324 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-07 16:54:08,324 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-07 16:54:08,327 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-07 16:54:08,328 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/info 2023-02-07 16:54:08,328 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/info 2023-02-07 16:54:08,329 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-07 16:54:08,330 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:08,330 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-07 16:54:08,331 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/rep_barrier 2023-02-07 16:54:08,332 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/rep_barrier 2023-02-07 16:54:08,332 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-07 16:54:08,333 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:08,333 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-07 16:54:08,334 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/table 2023-02-07 16:54:08,335 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/table 2023-02-07 16:54:08,335 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-07 16:54:08,336 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:08,338 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740 2023-02-07 16:54:08,341 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740 2023-02-07 16:54:08,349 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-07 16:54:08,352 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-07 16:54:08,353 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64673608, jitterRate=-0.03628814220428467}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-07 16:54:08,353 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675788848324Writing region info on filesystem at 1675788848324Initializing all the Stores at 1675788848326 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675788848326Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788848326Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675788848326Cleaning up temporary data from old regions at 1675788848345 (+19 ms)Cleaning up detritus from prior splits at 1675788848347 (+2 ms)Running coprocessor post-open hooks at 1675788848353 (+6 ms)Region opened successfully at 1675788848353 2023-02-07 16:54:08,362 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675788848235 2023-02-07 16:54:08,379 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-07 16:54:08,380 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-07 16:54:08,381 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,40593,1675788846388, state=OPEN 2023-02-07 16:54:08,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-07 16:54:08,391 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-07 16:54:08,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-07 16:54:08,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,40593,1675788846388 in 331 msec 2023-02-07 16:54:08,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-07 16:54:08,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 568 msec 2023-02-07 16:54:08,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 855 msec 2023-02-07 16:54:08,410 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675788846544, completionTime=-1 2023-02-07 16:54:08,410 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-07 16:54:08,410 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:08,477 DEBUG [hconnection-0xa97c7a1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 16:54:08,480 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:36940, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 16:54:08,497 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:08,497 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788908497 2023-02-07 16:54:08,497 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788968497 2023-02-07 16:54:08,497 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 86 msec 2023-02-07 16:54:08,541 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39383,1675788844945-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,542 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39383,1675788844945-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,542 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39383,1675788844945-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,543 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase12:39383, period=300000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,543 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-07 16:54:08,554 DEBUG [master/jenkins-hbase12:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-07 16:54:08,561 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-07 16:54:08,562 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-07 16:54:08,577 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:08,580 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:08,584 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:08,609 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:08,611 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197 empty. 2023-02-07 16:54:08,612 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:08,612 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-07 16:54:08,655 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:08,658 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 602ba3a55336ac4b5e29ccd78323a197, NAME => 'hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:08,676 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:08,676 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 602ba3a55336ac4b5e29ccd78323a197, disabling compactions & flushes 2023-02-07 16:54:08,676 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:08,676 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:08,677 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. after waiting 0 ms 2023-02-07 16:54:08,677 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:08,677 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:08,677 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 602ba3a55336ac4b5e29ccd78323a197: Waiting for close lock at 1675788848676Disabling compacts and flushes for region at 1675788848676Disabling writes for close at 1675788848677 (+1 ms)Writing region close event to WAL at 1675788848677Closed at 1675788848677 2023-02-07 16:54:08,683 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:08,697 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675788848686"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788848686"}]},"ts":"1675788848686"} 2023-02-07 16:54:08,722 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:08,724 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:08,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788848724"}]},"ts":"1675788848724"} 2023-02-07 16:54:08,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-07 16:54:08,750 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:08,751 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:08,751 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:08,751 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:08,751 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:08,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=602ba3a55336ac4b5e29ccd78323a197, ASSIGN}] 2023-02-07 16:54:08,759 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=602ba3a55336ac4b5e29ccd78323a197, ASSIGN 2023-02-07 16:54:08,761 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=602ba3a55336ac4b5e29ccd78323a197, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=false 2023-02-07 16:54:08,913 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:08,914 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=602ba3a55336ac4b5e29ccd78323a197, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:08,914 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675788848914"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788848914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788848914"}]},"ts":"1675788848914"} 2023-02-07 16:54:08,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 602ba3a55336ac4b5e29ccd78323a197, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:09,075 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:09,075 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 16:54:09,078 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:45856, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 16:54:09,084 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:09,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 602ba3a55336ac4b5e29ccd78323a197, NAME => 'hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:09,086 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:09,087 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,087 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:09,087 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,087 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,089 INFO [StoreOpener-602ba3a55336ac4b5e29ccd78323a197-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,093 DEBUG [StoreOpener-602ba3a55336ac4b5e29ccd78323a197-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/info 2023-02-07 16:54:09,093 DEBUG [StoreOpener-602ba3a55336ac4b5e29ccd78323a197-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/info 2023-02-07 16:54:09,094 INFO [StoreOpener-602ba3a55336ac4b5e29ccd78323a197-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 602ba3a55336ac4b5e29ccd78323a197 columnFamilyName info 2023-02-07 16:54:09,098 INFO [StoreOpener-602ba3a55336ac4b5e29ccd78323a197-1] regionserver.HStore(310): Store=602ba3a55336ac4b5e29ccd78323a197/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:09,100 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,105 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:54:09,109 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:09,111 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 602ba3a55336ac4b5e29ccd78323a197; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64642837, jitterRate=-0.036746665835380554}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 16:54:09,111 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 602ba3a55336ac4b5e29ccd78323a197: Running coprocessor pre-open hook at 1675788849087Writing region info on filesystem at 1675788849087Initializing all the Stores at 1675788849089 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675788849089Cleaning up temporary data from old regions at 1675788849102 (+13 ms)Cleaning up detritus from prior splits at 1675788849103 (+1 ms)Running coprocessor post-open hooks at 1675788849111 (+8 ms)Region opened successfully at 1675788849111 2023-02-07 16:54:09,113 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197., pid=6, masterSystemTime=1675788849074 2023-02-07 16:54:09,118 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:09,119 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:54:09,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:09,122 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675788849120"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788849120"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788849120"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788849120"}]},"ts":"1675788849120"} 2023-02-07 16:54:09,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-07 16:54:09,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 602ba3a55336ac4b5e29ccd78323a197, server=jenkins-hbase12.apache.org,43011,1675788846300 in 205 msec 2023-02-07 16:54:09,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-07 16:54:09,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=602ba3a55336ac4b5e29ccd78323a197, ASSIGN in 377 msec 2023-02-07 16:54:09,137 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:09,137 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788849137"}]},"ts":"1675788849137"} 2023-02-07 16:54:09,139 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-07 16:54:09,182 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-07 16:54:09,183 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:09,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 620 msec 2023-02-07 16:54:09,191 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-07 16:54:09,191 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:09,211 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 16:54:09,215 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:45872, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 16:54:09,227 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:09,254 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-07 16:54:09,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 50 msec 2023-02-07 16:54:09,281 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:09,296 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-07 16:54:09,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 30 msec 2023-02-07 16:54:09,339 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-07 16:54:09,360 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-07 16:54:09,360 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.799sec 2023-02-07 16:54:09,363 INFO [master/jenkins-hbase12:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-07 16:54:09,365 INFO [master/jenkins-hbase12:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-07 16:54:09,365 INFO [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-07 16:54:09,366 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39383,1675788844945-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-07 16:54:09,368 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39383,1675788844945-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-07 16:54:09,378 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-07 16:54:09,398 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ReadOnlyZKClient(139): Connect 0x42379693 to 127.0.0.1:50517 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-07 16:54:09,413 DEBUG [Listener at localhost.localdomain/34435] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c9c45f5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-07 16:54:09,427 DEBUG [hconnection-0x3e7d9739-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 16:54:09,439 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:36952, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 16:54:09,448 INFO [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:54:09,482 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=174, AvailableMemoryMB=3308 2023-02-07 16:54:09,486 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:09,487 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:09,487 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:09,494 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-07 16:54:09,498 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:51346, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-07 16:54:09,528 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:54:09,528 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:54:09,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:54:09,535 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:09,538 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:09,540 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:09,543 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,544 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 empty. 2023-02-07 16:54:09,547 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,547 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-07 16:54:09,570 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:09,572 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => ec19d20db6bb6b93bd46e1c22dcf6fe3, NAME => 'testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:09,595 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:09,595 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing ec19d20db6bb6b93bd46e1c22dcf6fe3, disabling compactions & flushes 2023-02-07 16:54:09,595 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,595 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,595 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. after waiting 0 ms 2023-02-07 16:54:09,595 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,595 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,595 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: Waiting for close lock at 1675788849595Disabling compacts and flushes for region at 1675788849595Disabling writes for close at 1675788849595Writing region close event to WAL at 1675788849595Closed at 1675788849595 2023-02-07 16:54:09,600 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:09,603 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675788849602"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788849602"}]},"ts":"1675788849602"} 2023-02-07 16:54:09,606 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:09,608 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:09,608 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788849608"}]},"ts":"1675788849608"} 2023-02-07 16:54:09,611 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-07 16:54:09,634 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:09,636 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:09,636 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:09,636 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:09,636 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:09,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, ASSIGN}] 2023-02-07 16:54:09,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, ASSIGN 2023-02-07 16:54:09,642 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:54:09,792 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:09,793 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ec19d20db6bb6b93bd46e1c22dcf6fe3, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:09,794 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675788849793"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788849793"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788849793"}]},"ts":"1675788849793"} 2023-02-07 16:54:09,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure ec19d20db6bb6b93bd46e1c22dcf6fe3, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:09,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ec19d20db6bb6b93bd46e1c22dcf6fe3, NAME => 'testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:09,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:09,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:09,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,960 INFO [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,962 DEBUG [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1 2023-02-07 16:54:09,962 DEBUG [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1 2023-02-07 16:54:09,963 INFO [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec19d20db6bb6b93bd46e1c22dcf6fe3 columnFamilyName cf1 2023-02-07 16:54:09,963 INFO [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] regionserver.HStore(310): Store=ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:09,963 INFO [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,965 DEBUG [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2 2023-02-07 16:54:09,965 DEBUG [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2 2023-02-07 16:54:09,966 INFO [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec19d20db6bb6b93bd46e1c22dcf6fe3 columnFamilyName cf2 2023-02-07 16:54:09,967 INFO [StoreOpener-ec19d20db6bb6b93bd46e1c22dcf6fe3-1] regionserver.HStore(310): Store=ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:09,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:09,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:09,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:09,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened ec19d20db6bb6b93bd46e1c22dcf6fe3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60987007, jitterRate=-0.09122277796268463}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:09,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: Running coprocessor pre-open hook at 1675788849958Writing region info on filesystem at 1675788849958Initializing all the Stores at 1675788849960 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788849960Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788849960Cleaning up temporary data from old regions at 1675788849971 (+11 ms)Cleaning up detritus from prior splits at 1675788849972 (+1 ms)Running coprocessor post-open hooks at 1675788849980 (+8 ms)Region opened successfully at 1675788849980 2023-02-07 16:54:09,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3., pid=11, masterSystemTime=1675788849950 2023-02-07 16:54:09,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:09,985 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=ec19d20db6bb6b93bd46e1c22dcf6fe3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:09,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675788849985"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788849985"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788849985"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788849985"}]},"ts":"1675788849985"} 2023-02-07 16:54:09,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-07 16:54:09,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure ec19d20db6bb6b93bd46e1c22dcf6fe3, server=jenkins-hbase12.apache.org,40593,1675788846388 in 191 msec 2023-02-07 16:54:09,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-07 16:54:09,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, ASSIGN in 355 msec 2023-02-07 16:54:09,997 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:09,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788849997"}]},"ts":"1675788849997"} 2023-02-07 16:54:09,999 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-07 16:54:10,014 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:10,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 481 msec 2023-02-07 16:54:10,086 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:10,088 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:10,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:10,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,099 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:10,099 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,100 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 16:54:10,100 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 16:54:10,101 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,101 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,107 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,107 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,107 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,107 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,108 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,108 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:10,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,111 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,111 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,111 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,111 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,112 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,113 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,113 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,113 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,114 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,114 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,113 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,114 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:10,114 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:10,114 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:10,114 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. started... 2023-02-07 16:54:10,116 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:54:10,127 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,127 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,128 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,128 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,128 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,128 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,190 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/904e3a96f8f9460c9bd4699d8941d305 2023-02-07 16:54:10,255 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/3ed03880fe2646cb9588bd25b5cca535 2023-02-07 16:54:10,266 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/904e3a96f8f9460c9bd4699d8941d305 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305 2023-02-07 16:54:10,277 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:10,281 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/3ed03880fe2646cb9588bd25b5cca535 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535 2023-02-07 16:54:10,291 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:10,293 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 177ms, sequenceid=6, compaction requested=false 2023-02-07 16:54:10,294 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:10,296 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: 2023-02-07 16:54:10,296 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:10,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:10,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:10,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,298 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,298 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,307 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,307 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:10,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:10,308 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,308 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:10,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:10,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:10,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 16:54:10,311 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:10,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@774c92e[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:10,311 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,345 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,346 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:10,346 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:10,361 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:10,362 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,362 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:10,370 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:10,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,370 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:10,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:10,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:10,370 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,370 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,371 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:10,371 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:10,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:10,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:10,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:10,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:54:10,377 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 16:54:10,377 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:10,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:10,377 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:10,377 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,377 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:10,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:10,392 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,393 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:10,393 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:10,395 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,395 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,396 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,413 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:10,413 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,413 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,413 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:10,413 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,413 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,413 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,414 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,414 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,414 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,414 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,415 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,416 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:10,416 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:10,516 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:10,516 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:10,517 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:10,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:10,547 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:10,547 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:10,548 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:10,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:10,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,551 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:10,551 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,552 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 16:54:10,552 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 16:54:10,553 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,553 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,560 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,560 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,560 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,560 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,560 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,560 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:10,562 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,561 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,562 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,562 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,563 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,563 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,564 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,564 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. started... 2023-02-07 16:54:10,564 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,564 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:10,570 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,581 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,581 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,582 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,586 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/ad44d9ba843447d3b1454d8cd12e8027 2023-02-07 16:54:10,614 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/e7ae945734714f33ad441bf5bc2e73c0 2023-02-07 16:54:10,623 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/ad44d9ba843447d3b1454d8cd12e8027 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027 2023-02-07 16:54:10,632 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:10,633 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/e7ae945734714f33ad441bf5bc2e73c0 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0 2023-02-07 16:54:10,641 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:10,643 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 78ms, sequenceid=19, compaction requested=false 2023-02-07 16:54:10,643 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: 2023-02-07 16:54:10,643 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:10,643 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:10,643 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:10,644 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,644 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,644 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,654 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:10,655 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,655 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 16:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44325e9[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:10,657 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:10,657 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,665 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:10,665 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,665 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:10,665 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:10,665 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:10,666 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:10,666 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:10,666 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:10,666 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:10,676 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:10,676 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,676 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:10,686 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:10,686 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,686 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:10,686 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:10,696 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:10,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:10,696 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,696 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:54:10,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:10,701 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 16:54:10,701 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:10,701 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:10,701 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,702 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,708 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:10,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,709 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:10,709 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,711 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,712 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,712 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,756 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,756 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,756 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,757 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,757 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,758 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:10,758 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,758 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,758 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:10,759 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:10,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:10,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:10,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,759 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:10,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,859 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:10,859 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:10,859 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:10,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:10,886 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:10,887 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:10,888 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:10,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:10,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,890 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:10,890 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,891 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 16:54:10,891 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 16:54:10,892 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,892 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:10,907 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,909 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:10,909 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,910 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,910 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:10,910 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,910 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,910 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,910 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:10,911 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:10,911 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:10,911 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. started... 2023-02-07 16:54:10,912 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:10,917 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,928 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,928 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:10,928 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:10,929 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/7d21dd3bdb3447e1a6e1796a58a83275 2023-02-07 16:54:10,955 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/14c1c54491a04325af2f0e986d8f00cd 2023-02-07 16:54:10,965 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/7d21dd3bdb3447e1a6e1796a58a83275 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275 2023-02-07 16:54:10,974 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:10,976 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/14c1c54491a04325af2f0e986d8f00cd as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd 2023-02-07 16:54:10,985 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:10,986 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 74ms, sequenceid=32, compaction requested=true 2023-02-07 16:54:10,986 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: 2023-02-07 16:54:10,986 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:10,987 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:10,987 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:10,987 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:10,987 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:10,987 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,033 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 16:54:11,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cc5bcc3[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:11,036 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:11,036 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,124 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,124 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:11,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:11,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,125 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,125 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,125 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,125 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,125 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:11,125 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:11,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,223 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,223 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,223 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,223 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,223 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,223 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,224 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:54:11,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:11,229 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 16:54:11,229 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:11,229 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:11,229 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,229 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:11,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,234 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,234 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,234 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,235 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,235 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,237 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,237 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,237 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,244 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,245 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,244 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:11,245 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:11,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,245 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,245 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,245 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,245 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:11,245 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:11,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,246 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,346 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:11,346 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:11,346 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:11,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:11,372 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:11,372 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:11,373 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:11,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:11,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,375 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:11,375 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,376 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 16:54:11,376 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 16:54:11,376 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,377 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,380 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,380 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,380 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,380 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,381 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,381 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,383 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:11,383 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:11,383 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:11,383 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,383 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,383 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,383 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:11,383 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,383 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,383 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:11,384 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,384 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:11,385 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:11,385 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. started... 2023-02-07 16:54:11,385 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:11,391 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,391 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,392 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,392 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,392 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,392 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,402 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/2984f29718ef4084aff4cd7e51db8d0d 2023-02-07 16:54:11,433 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/ef43a110a795496284ec568c1cf088a8 2023-02-07 16:54:11,440 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/2984f29718ef4084aff4cd7e51db8d0d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d 2023-02-07 16:54:11,448 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:11,450 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/ef43a110a795496284ec568c1cf088a8 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8 2023-02-07 16:54:11,458 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:11,459 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 74ms, sequenceid=45, compaction requested=true 2023-02-07 16:54:11,459 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: 2023-02-07 16:54:11,460 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:11,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:11,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:11,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:11,461 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,536 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,537 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,537 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,537 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 16:54:11,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5708d210[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:11,540 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:11,540 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,601 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,601 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,601 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:11,602 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,602 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:11,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:11,602 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:11,612 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,612 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,612 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,612 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,623 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,623 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,623 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,633 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,633 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,633 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:54:11,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:11,638 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 16:54:11,638 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:11,638 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:11,638 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,638 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,644 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,645 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,646 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,647 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,647 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,648 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,654 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,655 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,654 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:11,655 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:11,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,655 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,655 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:11,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,656 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,756 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:11,757 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:11,757 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:11,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:11,782 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:11,782 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:11,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:11,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:11,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,786 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:11,786 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 16:54:11,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 16:54:11,788 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,788 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,791 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,791 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,791 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,791 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,792 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,792 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:11,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,793 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,793 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,793 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:11,793 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,793 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:11,793 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:11,793 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:11,793 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,796 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:11,796 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,795 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,796 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:11,797 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:11,797 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:11,796 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:11,797 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,797 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:11,797 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:11,799 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:11,799 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:11,799 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:11,799 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:11,799 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. started... 2023-02-07 16:54:11,799 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,800 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:11,813 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,819 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/c329f1365a6a45a08ee61744f8ab41f7 2023-02-07 16:54:11,823 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,823 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,823 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,823 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,823 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,844 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/a05dc510a4bb47739d7a197ffd07a6a4 2023-02-07 16:54:11,852 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/c329f1365a6a45a08ee61744f8ab41f7 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7 2023-02-07 16:54:11,860 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:11,862 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/a05dc510a4bb47739d7a197ffd07a6a4 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4 2023-02-07 16:54:11,870 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:11,871 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 71ms, sequenceid=58, compaction requested=true 2023-02-07 16:54:11,871 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: 2023-02-07 16:54:11,871 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:11,871 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:11,872 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:11,872 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,872 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:11,872 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:11,885 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,885 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,886 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,886 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:11,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 16:54:11,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76945c66[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:11,889 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:11,889 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,896 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,896 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,896 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,897 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,897 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:11,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:11,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:11,897 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:11,907 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,907 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,907 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,917 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,917 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,918 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:11,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:11,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:54:11,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:11,934 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 16:54:11,934 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:11,934 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:11,934 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,934 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:11,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:11,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,939 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,939 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,939 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,940 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,940 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,941 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:11,942 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,942 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,980 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:11,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:11,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:11,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:11,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,980 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:11,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:11,981 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,081 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:12,081 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:12,081 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:12,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:12,108 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:12,108 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:12,108 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:12,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:12,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,111 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:12,111 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:12,112 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-07 16:54:12,112 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-07 16:54:12,112 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,112 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,138 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,139 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:12,139 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:12,139 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:12,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:12,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:12,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:12,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:12,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:12,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:12,139 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,139 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:12,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:12,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:12,141 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-07 16:54:12,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,141 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,141 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:12,142 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:12,142 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:12,142 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:12,142 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:12,141 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:12,142 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:12,142 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-07 16:54:12,143 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:12,143 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:12,143 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:12,143 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. started... 2023-02-07 16:54:12,143 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:12,149 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,149 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,150 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,150 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:12,151 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,151 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:12,162 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/5230681cbe7241dd8df391d16522a097 2023-02-07 16:54:12,182 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/82c95fb672d240cb9683c792156447db 2023-02-07 16:54:12,190 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/5230681cbe7241dd8df391d16522a097 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097 2023-02-07 16:54:12,197 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:12,199 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/82c95fb672d240cb9683c792156447db as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db 2023-02-07 16:54:12,207 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:12,208 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 65ms, sequenceid=71, compaction requested=true 2023-02-07 16:54:12,208 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: 2023-02-07 16:54:12,208 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:12,208 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:12,209 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:12,209 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,209 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-07 16:54:12,209 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-07 16:54:12,222 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:12,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:12,223 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,223 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:12,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:12,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-07 16:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b654330[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:12,225 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:12,225 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,233 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:12,233 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,234 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:12,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:12,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-07 16:54:12,234 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:12,244 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:12,244 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,244 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:12,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,244 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:12,254 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:12,254 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:12,254 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:12,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-07 16:54:12,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:12,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-07 16:54:12,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:12,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:54:12,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:12,269 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-07 16:54:12,269 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:12,269 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:12,269 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,269 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:12,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:12,276 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:12,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:12,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:12,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:12,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:12,276 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:12,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:12,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:12,277 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:12,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,279 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:12,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,281 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,281 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,281 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:12,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:12,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:12,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-07 16:54:12,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:12,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:12,286 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:12,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:12,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:12,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:12,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:12,287 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:12,287 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:12,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:12,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:12,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:12,287 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:12,287 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,288 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-07 16:54:12,388 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:12,388 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:12,388 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:12,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:12,414 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,421 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:12,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:12,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:12,422 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:12,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, UNASSIGN}] 2023-02-07 16:54:12,433 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, UNASSIGN 2023-02-07 16:54:12,434 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ec19d20db6bb6b93bd46e1c22dcf6fe3, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:12,434 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675788852434"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788852434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788852434"}]},"ts":"1675788852434"} 2023-02-07 16:54:12,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure ec19d20db6bb6b93bd46e1c22dcf6fe3, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:12,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing ec19d20db6bb6b93bd46e1c22dcf6fe3, disabling compactions & flushes 2023-02-07 16:54:12,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:12,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:12,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. after waiting 0 ms 2023-02-07 16:54:12,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:12,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:54:12,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/703556b45aa6439aa22052627e97e75f 2023-02-07 16:54:12,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/fb5d50dc0113470fada332a2f20912a1 2023-02-07 16:54:12,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf1/703556b45aa6439aa22052627e97e75f as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f 2023-02-07 16:54:12,657 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:54:12,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.tmp/cf2/fb5d50dc0113470fada332a2f20912a1 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1 2023-02-07 16:54:12,668 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:54:12,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 72ms, sequenceid=82, compaction requested=true 2023-02-07 16:54:12,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:54:12,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:12,690 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. 2023-02-07 16:54:12,690 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for ec19d20db6bb6b93bd46e1c22dcf6fe3: Waiting for close lock at 1675788852597Running coprocessor pre-close hooks at 1675788852597Disabling compacts and flushes for region at 1675788852597Disabling writes for close at 1675788852597Obtaining lock to block concurrent updates at 1675788852597Preparing flush snapshotting stores in ec19d20db6bb6b93bd46e1c22dcf6fe3 at 1675788852597Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675788852598 (+1 ms)Flushing stores of testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. at 1675788852598Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1: creating writer at 1675788852599 (+1 ms)Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1: appending metadata at 1675788852602 (+3 ms)Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1: closing flushed file at 1675788852602Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2: creating writer at 1675788852623 (+21 ms)Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2: appending metadata at 1675788852627 (+4 ms)Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2: closing flushed file at 1675788852627Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1: reopening flushed file at 1675788852651 (+24 ms)Flushing ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2: reopening flushed file at 1675788852662 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ec19d20db6bb6b93bd46e1c22dcf6fe3 in 72ms, sequenceid=82, compaction requested=true at 1675788852669 (+7 ms)Writing region close event to WAL at 1675788852684 (+15 ms)Running coprocessor post-close hooks at 1675788852689 (+5 ms)Closed at 1675788852690 (+1 ms) 2023-02-07 16:54:12,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,693 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=ec19d20db6bb6b93bd46e1c22dcf6fe3, regionState=CLOSED 2023-02-07 16:54:12,694 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675788852693"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788852693"}]},"ts":"1675788852693"} 2023-02-07 16:54:12,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-07 16:54:12,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure ec19d20db6bb6b93bd46e1c22dcf6fe3, server=jenkins-hbase12.apache.org,40593,1675788846388 in 259 msec 2023-02-07 16:54:12,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-07 16:54:12,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, UNASSIGN in 268 msec 2023-02-07 16:54:12,720 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=ec19d20db6bb6b93bd46e1c22dcf6fe3, threads=14 2023-02-07 16:54:12,721 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,721 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,721 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,722 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,722 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,722 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,722 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,722 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,722 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,723 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,723 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,723 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,723 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,723 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,752 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db, top=true 2023-02-07 16:54:12,752 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4, top=true 2023-02-07 16:54:12,807 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7, top=true 2023-02-07 16:54:12,809 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097, top=true 2023-02-07 16:54:12,810 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1, top=true 2023-02-07 16:54:12,814 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f, top=true 2023-02-07 16:54:12,829 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.splits/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db for child: a4f8504d73cbf3682ed9cc92ec5a306b, parent: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,829 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,841 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.splits/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4 for child: a4f8504d73cbf3682ed9cc92ec5a306b, parent: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,843 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,866 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.splits/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1 for child: a4f8504d73cbf3682ed9cc92ec5a306b, parent: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,867 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,882 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.splits/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f for child: a4f8504d73cbf3682ed9cc92ec5a306b, parent: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,883 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,884 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.splits/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097 for child: a4f8504d73cbf3682ed9cc92ec5a306b, parent: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,890 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,893 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/.splits/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7 for child: a4f8504d73cbf3682ed9cc92ec5a306b, parent: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,895 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,953 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,964 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,969 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,985 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:12,988 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:13,385 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:13,388 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:13,388 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275 for region: ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:13,388 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region ec19d20db6bb6b93bd46e1c22dcf6fe3 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:54:13,453 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:54:13,457 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:54:13,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675788853460"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675788853460"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675788853460"}]},"ts":"1675788853460"} 2023-02-07 16:54:13,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788853460"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788853460"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788853460"}]},"ts":"1675788853460"} 2023-02-07 16:54:13,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788853460"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788853460"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788853460"}]},"ts":"1675788853460"} 2023-02-07 16:54:13,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, ASSIGN}] 2023-02-07 16:54:13,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, ASSIGN 2023-02-07 16:54:13,508 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:54:13,510 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, ASSIGN 2023-02-07 16:54:13,511 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:54:13,658 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:54:13,660 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a4f8504d73cbf3682ed9cc92ec5a306b, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:13,660 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788853660"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788853660"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788853660"}]},"ts":"1675788853660"} 2023-02-07 16:54:13,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure a4f8504d73cbf3682ed9cc92ec5a306b, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:13,667 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=fb1efbdcd387292fcd3a5366f391724e, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:13,667 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788853667"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788853667"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788853667"}]},"ts":"1675788853667"} 2023-02-07 16:54:13,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure fb1efbdcd387292fcd3a5366f391724e, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:13,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fb1efbdcd387292fcd3a5366f391724e, NAME => 'testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.', STARTKEY => '', ENDKEY => '26'} 2023-02-07 16:54:13,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:13,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:13,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,823 INFO [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,825 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1 2023-02-07 16:54:13,825 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1 2023-02-07 16:54:13,826 INFO [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb1efbdcd387292fcd3a5366f391724e columnFamilyName cf1 2023-02-07 16:54:13,858 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:54:13,871 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d-bottom 2023-02-07 16:54:13,880 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275-bottom 2023-02-07 16:54:13,888 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305-bottom 2023-02-07 16:54:13,896 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027-bottom 2023-02-07 16:54:13,896 INFO [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(310): Store=fb1efbdcd387292fcd3a5366f391724e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:13,896 INFO [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,898 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2 2023-02-07 16:54:13,898 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2 2023-02-07 16:54:13,898 INFO [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb1efbdcd387292fcd3a5366f391724e columnFamilyName cf2 2023-02-07 16:54:13,930 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd-bottom 2023-02-07 16:54:13,938 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535-bottom 2023-02-07 16:54:13,940 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-07 16:54:13,942 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-07 16:54:13,948 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0-bottom 2023-02-07 16:54:13,957 DEBUG [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8-bottom 2023-02-07 16:54:13,957 INFO [StoreOpener-fb1efbdcd387292fcd3a5366f391724e-1] regionserver.HStore(310): Store=fb1efbdcd387292fcd3a5366f391724e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:13,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:13,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:13,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened fb1efbdcd387292fcd3a5366f391724e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66940741, jitterRate=-0.0025052279233932495}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:13,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for fb1efbdcd387292fcd3a5366f391724e: Running coprocessor pre-open hook at 1675788853821Writing region info on filesystem at 1675788853821Initializing all the Stores at 1675788853823 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788853823Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788853823Cleaning up temporary data from old regions at 1675788853961 (+138 ms)Cleaning up detritus from prior splits at 1675788853961Running coprocessor post-open hooks at 1675788853965 (+4 ms)Region opened successfully at 1675788853965 2023-02-07 16:54:13,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e., pid=18, masterSystemTime=1675788853815 2023-02-07 16:54:13,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:13,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:13,969 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-07 16:54:13,969 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-07 16:54:13,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:13,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a4f8504d73cbf3682ed9cc92ec5a306b, NAME => 'testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.', STARTKEY => '26', ENDKEY => ''} 2023-02-07 16:54:13,971 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,971 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,971 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1912): fb1efbdcd387292fcd3a5366f391724e/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:13,972 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1912): fb1efbdcd387292fcd3a5366f391724e/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:13,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:13,972 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fb1efbdcd387292fcd3a5366f391724e/cf2 in testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,972 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fb1efbdcd387292fcd3a5366f391724e/cf1 in testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:13,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:13,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=fb1efbdcd387292fcd3a5366f391724e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:13,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:13,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:13,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:13,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788853972"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788853972"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788853972"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788853972"}]},"ts":"1675788853972"} 2023-02-07 16:54:13,972 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/.tmp, totalSize=19.9 K 2023-02-07 16:54:13,972 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/.tmp, totalSize=19.9 K 2023-02-07 16:54:13,975 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788850041 2023-02-07 16:54:13,975 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788850041 2023-02-07 16:54:13,976 INFO [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:13,976 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788850523 2023-02-07 16:54:13,976 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788850523 2023-02-07 16:54:13,977 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788850862 2023-02-07 16:54:13,977 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1 2023-02-07 16:54:13,977 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1 2023-02-07 16:54:13,977 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788850862 2023-02-07 16:54:13,977 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788851349 2023-02-07 16:54:13,978 INFO [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4f8504d73cbf3682ed9cc92ec5a306b columnFamilyName cf1 2023-02-07 16:54:13,978 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788851349 2023-02-07 16:54:13,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-07 16:54:13,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure fb1efbdcd387292fcd3a5366f391724e, server=jenkins-hbase12.apache.org,40593,1675788846388 in 307 msec 2023-02-07 16:54:13,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, ASSIGN in 476 msec 2023-02-07 16:54:14,000 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] throttle.PressureAwareThroughputController(145): fb1efbdcd387292fcd3a5366f391724e#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:14,001 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d-top 2023-02-07 16:54:14,010 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275-top 2023-02-07 16:54:14,016 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] throttle.PressureAwareThroughputController(145): fb1efbdcd387292fcd3a5366f391724e#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:14,017 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305-top 2023-02-07 16:54:14,031 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027-top 2023-02-07 16:54:14,039 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/.tmp/cf2/4d71b3a2c08345fda970dda3118eb553 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/4d71b3a2c08345fda970dda3118eb553 2023-02-07 16:54:14,039 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097 2023-02-07 16:54:14,050 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f 2023-02-07 16:54:14,058 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7 2023-02-07 16:54:14,059 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fb1efbdcd387292fcd3a5366f391724e/cf2 of fb1efbdcd387292fcd3a5366f391724e into 4d71b3a2c08345fda970dda3118eb553(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:14,059 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fb1efbdcd387292fcd3a5366f391724e: 2023-02-07 16:54:14,059 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e., storeName=fb1efbdcd387292fcd3a5366f391724e/cf2, priority=12, startTime=1675788853967; duration=0sec 2023-02-07 16:54:14,060 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:14,061 INFO [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(310): Store=a4f8504d73cbf3682ed9cc92ec5a306b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:14,061 INFO [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:14,062 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2 2023-02-07 16:54:14,062 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2 2023-02-07 16:54:14,063 INFO [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4f8504d73cbf3682ed9cc92ec5a306b columnFamilyName cf2 2023-02-07 16:54:14,063 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/.tmp/cf1/08e17b173ae64cc8996d18403e62f237 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/08e17b173ae64cc8996d18403e62f237 2023-02-07 16:54:14,071 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fb1efbdcd387292fcd3a5366f391724e/cf1 of fb1efbdcd387292fcd3a5366f391724e into 08e17b173ae64cc8996d18403e62f237(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:14,071 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fb1efbdcd387292fcd3a5366f391724e: 2023-02-07 16:54:14,071 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e., storeName=fb1efbdcd387292fcd3a5366f391724e/cf1, priority=12, startTime=1675788853967; duration=0sec 2023-02-07 16:54:14,071 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:14,079 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd-top 2023-02-07 16:54:14,085 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535-top 2023-02-07 16:54:14,091 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0-top 2023-02-07 16:54:14,097 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8-top 2023-02-07 16:54:14,104 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db 2023-02-07 16:54:14,110 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4 2023-02-07 16:54:14,116 DEBUG [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1 2023-02-07 16:54:14,116 INFO [StoreOpener-a4f8504d73cbf3682ed9cc92ec5a306b-1] regionserver.HStore(310): Store=a4f8504d73cbf3682ed9cc92ec5a306b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:14,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:14,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:14,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:14,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:14,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened a4f8504d73cbf3682ed9cc92ec5a306b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60132067, jitterRate=-0.10396237671375275}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:14,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for a4f8504d73cbf3682ed9cc92ec5a306b: Running coprocessor pre-open hook at 1675788853972Writing region info on filesystem at 1675788853972Initializing all the Stores at 1675788853975 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788853976 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788853976Cleaning up temporary data from old regions at 1675788854120 (+144 ms)Cleaning up detritus from prior splits at 1675788854120Running coprocessor post-open hooks at 1675788854125 (+5 ms)Region opened successfully at 1675788854125 2023-02-07 16:54:14,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b., pid=17, masterSystemTime=1675788853815 2023-02-07 16:54:14,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:14,127 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:14,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:14,127 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:14,127 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,127 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1912): a4f8504d73cbf3682ed9cc92ec5a306b/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:14,127 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,127 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1912): a4f8504d73cbf3682ed9cc92ec5a306b/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:14,127 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a4f8504d73cbf3682ed9cc92ec5a306b/cf1 in testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,128 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a4f8504d73cbf3682ed9cc92ec5a306b/cf2 in testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,128 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/.tmp, totalSize=34.9 K 2023-02-07 16:54:14,128 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/.tmp, totalSize=34.9 K 2023-02-07 16:54:14,130 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788850041 2023-02-07 16:54:14,130 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788850041 2023-02-07 16:54:14,131 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788850523 2023-02-07 16:54:14,131 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788850523 2023-02-07 16:54:14,132 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788850862 2023-02-07 16:54:14,132 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a4f8504d73cbf3682ed9cc92ec5a306b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:14,132 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788851349 2023-02-07 16:54:14,132 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788854132"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788854132"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788854132"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788854132"}]},"ts":"1675788854132"} 2023-02-07 16:54:14,133 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788850862 2023-02-07 16:54:14,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,133 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788851759 2023-02-07 16:54:14,133 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788851349 2023-02-07 16:54:14,134 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788852084 2023-02-07 16:54:14,134 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788851759 2023-02-07 16:54:14,135 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788852084 2023-02-07 16:54:14,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,134 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788852391 2023-02-07 16:54:14,135 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788852391 2023-02-07 16:54:14,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-07 16:54:14,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure a4f8504d73cbf3682ed9cc92ec5a306b, server=jenkins-hbase12.apache.org,40593,1675788846388 in 474 msec 2023-02-07 16:54:14,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-07 16:54:14,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, ASSIGN in 637 msec 2023-02-07 16:54:14,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b in 1.7320 sec 2023-02-07 16:54:14,171 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] throttle.PressureAwareThroughputController(145): a4f8504d73cbf3682ed9cc92ec5a306b#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:14,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40593] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:36952 deadline: 1675788914176, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3. is not online on jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:14,179 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] throttle.PressureAwareThroughputController(145): a4f8504d73cbf3682ed9cc92ec5a306b#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:14,206 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/.tmp/cf2/65dd5a4f0f104f2c9df7454dd35b6928 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/65dd5a4f0f104f2c9df7454dd35b6928 2023-02-07 16:54:14,209 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/.tmp/cf1/fc932cf432874b13bba7f1f6c3ef21fa as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/fc932cf432874b13bba7f1f6c3ef21fa 2023-02-07 16:54:14,214 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a4f8504d73cbf3682ed9cc92ec5a306b/cf2 of a4f8504d73cbf3682ed9cc92ec5a306b into 65dd5a4f0f104f2c9df7454dd35b6928(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:14,214 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a4f8504d73cbf3682ed9cc92ec5a306b: 2023-02-07 16:54:14,214 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b., storeName=a4f8504d73cbf3682ed9cc92ec5a306b/cf2, priority=9, startTime=1675788854127; duration=0sec 2023-02-07 16:54:14,214 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:14,215 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a4f8504d73cbf3682ed9cc92ec5a306b/cf1 of a4f8504d73cbf3682ed9cc92ec5a306b into fc932cf432874b13bba7f1f6c3ef21fa(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:14,215 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a4f8504d73cbf3682ed9cc92ec5a306b: 2023-02-07 16:54:14,215 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b., storeName=a4f8504d73cbf3682ed9cc92ec5a306b/cf1, priority=9, startTime=1675788854126; duration=0sec 2023-02-07 16:54:14,216 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:14,316 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-07 16:54:14,331 INFO [Listener at localhost.localdomain/34435] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 16:54:14,334 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-07 16:54:14,381 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:14,382 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:14,382 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:14,392 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:14,392 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:14,393 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-07 16:54:14,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionUnevenDaughter 2023-02-07 16:54:14,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:14,413 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788854413"}]},"ts":"1675788854413"} 2023-02-07 16:54:14,415 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-07 16:54:14,417 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:14,417 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:14,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 16:54:14,475 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-07 16:54:14,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, UNASSIGN}] 2023-02-07 16:54:14,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, UNASSIGN 2023-02-07 16:54:14,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a4f8504d73cbf3682ed9cc92ec5a306b, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:14,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788854480"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788854480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788854480"}]},"ts":"1675788854480"} 2023-02-07 16:54:14,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure a4f8504d73cbf3682ed9cc92ec5a306b, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:14,486 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, UNASSIGN 2023-02-07 16:54:14,487 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fb1efbdcd387292fcd3a5366f391724e, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:14,487 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788854487"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788854487"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788854487"}]},"ts":"1675788854487"} 2023-02-07 16:54:14,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure fb1efbdcd387292fcd3a5366f391724e, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:14,522 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:14,522 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:14,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 16:54:14,635 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing a4f8504d73cbf3682ed9cc92ec5a306b, disabling compactions & flushes 2023-02-07 16:54:14,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. after waiting 0 ms 2023-02-07 16:54:14,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,646 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f] to archive 2023-02-07 16:54:14,647 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:14,652 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,654 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,656 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,658 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,660 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-c329f1365a6a45a08ee61744f8ab41f7 2023-02-07 16:54:14,663 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-5230681cbe7241dd8df391d16522a097 2023-02-07 16:54:14,664 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-703556b45aa6439aa22052627e97e75f 2023-02-07 16:54:14,687 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1] to archive 2023-02-07 16:54:14,688 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:14,690 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,692 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,694 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,696 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,698 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-a05dc510a4bb47739d7a197ffd07a6a4 2023-02-07 16:54:14,700 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-82c95fb672d240cb9683c792156447db 2023-02-07 16:54:14,702 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/testSplitTableRegionUnevenDaughter=ec19d20db6bb6b93bd46e1c22dcf6fe3-fb5d50dc0113470fada332a2f20912a1 2023-02-07 16:54:14,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:54:14,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:14,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b. 2023-02-07 16:54:14,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for a4f8504d73cbf3682ed9cc92ec5a306b: Waiting for close lock at 1675788854636Running coprocessor pre-close hooks at 1675788854636Disabling compacts and flushes for region at 1675788854636Disabling writes for close at 1675788854636Writing region close event to WAL at 1675788854703 (+67 ms)Running coprocessor post-close hooks at 1675788854708 (+5 ms)Closed at 1675788854708 2023-02-07 16:54:14,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:14,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:14,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing fb1efbdcd387292fcd3a5366f391724e, disabling compactions & flushes 2023-02-07 16:54:14,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:14,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:14,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. after waiting 0 ms 2023-02-07 16:54:14,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:14,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a4f8504d73cbf3682ed9cc92ec5a306b, regionState=CLOSED 2023-02-07 16:54:14,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788854712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788854712"}]},"ts":"1675788854712"} 2023-02-07 16:54:14,713 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d-bottom] to archive 2023-02-07 16:54:14,714 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:14,718 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/904e3a96f8f9460c9bd4699d8941d305.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-07 16:54:14,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure a4f8504d73cbf3682ed9cc92ec5a306b, server=jenkins-hbase12.apache.org,40593,1675788846388 in 233 msec 2023-02-07 16:54:14,720 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/ad44d9ba843447d3b1454d8cd12e8027.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a4f8504d73cbf3682ed9cc92ec5a306b, UNASSIGN in 243 msec 2023-02-07 16:54:14,723 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/7d21dd3bdb3447e1a6e1796a58a83275.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,724 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:14,724 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:14,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 16:54:14,725 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/2984f29718ef4084aff4cd7e51db8d0d.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,728 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8-bottom] to archive 2023-02-07 16:54:14,729 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:14,731 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/3ed03880fe2646cb9588bd25b5cca535.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,733 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/e7ae945734714f33ad441bf5bc2e73c0.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,735 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/14c1c54491a04325af2f0e986d8f00cd.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,737 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/ef43a110a795496284ec568c1cf088a8.ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:14,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:54:14,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:14,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e. 2023-02-07 16:54:14,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for fb1efbdcd387292fcd3a5366f391724e: Waiting for close lock at 1675788854711Running coprocessor pre-close hooks at 1675788854711Disabling compacts and flushes for region at 1675788854711Disabling writes for close at 1675788854712 (+1 ms)Writing region close event to WAL at 1675788854738 (+26 ms)Running coprocessor post-close hooks at 1675788854742 (+4 ms)Closed at 1675788854744 (+2 ms) 2023-02-07 16:54:14,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:14,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fb1efbdcd387292fcd3a5366f391724e, regionState=CLOSED 2023-02-07 16:54:14,748 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675788854747"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788854747"}]},"ts":"1675788854747"} 2023-02-07 16:54:14,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-07 16:54:14,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure fb1efbdcd387292fcd3a5366f391724e, server=jenkins-hbase12.apache.org,40593,1675788846388 in 261 msec 2023-02-07 16:54:14,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-07 16:54:14,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb1efbdcd387292fcd3a5366f391724e, UNASSIGN in 277 msec 2023-02-07 16:54:14,758 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788854758"}]},"ts":"1675788854758"} 2023-02-07 16:54:14,761 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-07 16:54:14,822 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-07 16:54:14,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 423 msec 2023-02-07 16:54:15,026 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,026 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-07 16:54:15,027 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-07 16:54:15,028 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,028 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,039 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,042 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,042 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,042 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-07 16:54:15,050 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:15,050 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:15,051 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:15,054 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/recovered.edits] 2023-02-07 16:54:15,054 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/recovered.edits] 2023-02-07 16:54:15,054 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/recovered.edits] 2023-02-07 16:54:15,057 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/08e17b173ae64cc8996d18403e62f237 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf1/08e17b173ae64cc8996d18403e62f237 2023-02-07 16:54:15,057 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/fc932cf432874b13bba7f1f6c3ef21fa to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf1/fc932cf432874b13bba7f1f6c3ef21fa 2023-02-07 16:54:15,058 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/2984f29718ef4084aff4cd7e51db8d0d 2023-02-07 16:54:15,060 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/5230681cbe7241dd8df391d16522a097 2023-02-07 16:54:15,061 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/65dd5a4f0f104f2c9df7454dd35b6928 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/cf2/65dd5a4f0f104f2c9df7454dd35b6928 2023-02-07 16:54:15,061 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/4d71b3a2c08345fda970dda3118eb553 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/cf2/4d71b3a2c08345fda970dda3118eb553 2023-02-07 16:54:15,067 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/703556b45aa6439aa22052627e97e75f 2023-02-07 16:54:15,069 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/7d21dd3bdb3447e1a6e1796a58a83275 2023-02-07 16:54:15,069 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e/recovered.edits/90.seqid 2023-02-07 16:54:15,070 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb1efbdcd387292fcd3a5366f391724e 2023-02-07 16:54:15,070 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b/recovered.edits/90.seqid 2023-02-07 16:54:15,070 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/904e3a96f8f9460c9bd4699d8941d305 2023-02-07 16:54:15,071 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:15,072 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/ad44d9ba843447d3b1454d8cd12e8027 2023-02-07 16:54:15,073 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf1/c329f1365a6a45a08ee61744f8ab41f7 2023-02-07 16:54:15,076 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/14c1c54491a04325af2f0e986d8f00cd 2023-02-07 16:54:15,078 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/3ed03880fe2646cb9588bd25b5cca535 2023-02-07 16:54:15,079 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/82c95fb672d240cb9683c792156447db 2023-02-07 16:54:15,081 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/a05dc510a4bb47739d7a197ffd07a6a4 2023-02-07 16:54:15,082 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/e7ae945734714f33ad441bf5bc2e73c0 2023-02-07 16:54:15,084 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/ef43a110a795496284ec568c1cf088a8 2023-02-07 16:54:15,086 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/cf2/fb5d50dc0113470fada332a2f20912a1 2023-02-07 16:54:15,089 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/recovered.edits/85.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3/recovered.edits/85.seqid 2023-02-07 16:54:15,090 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionUnevenDaughter/ec19d20db6bb6b93bd46e1c22dcf6fe3 2023-02-07 16:54:15,090 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-07 16:54:15,093 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,116 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-07 16:54:15,125 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-07 16:54:15,126 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-07 16:54:15,126 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:15,126 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-07 16:54:15,143 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,144 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-07 16:54:15,157 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-07 16:54:15,159 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,159 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-07 16:54:15,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788855159"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:15,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788855159"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:15,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788855159"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:15,163 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:54:15,163 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ec19d20db6bb6b93bd46e1c22dcf6fe3, NAME => 'testSplitTableRegionUnevenDaughter,,1675788849533.ec19d20db6bb6b93bd46e1c22dcf6fe3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fb1efbdcd387292fcd3a5366f391724e, NAME => 'testSplitTableRegionUnevenDaughter,,1675788852414.fb1efbdcd387292fcd3a5366f391724e.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => a4f8504d73cbf3682ed9cc92ec5a306b, NAME => 'testSplitTableRegionUnevenDaughter,26,1675788852414.a4f8504d73cbf3682ed9cc92ec5a306b.', STARTKEY => '26', ENDKEY => ''}] 2023-02-07 16:54:15,163 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-07 16:54:15,163 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788855163"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:15,166 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-07 16:54:15,255 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:15,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 223 msec 2023-02-07 16:54:15,345 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,345 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-07 16:54:15,347 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-07 16:54:15,363 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 360) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:35136 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase12:40593-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:37042 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:37116 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:35268 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:37146 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:60620 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:60300 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:35274 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:35112 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:60604 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:36966 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:37094 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:35154 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:37384 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:60618 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1936458839_15 at /127.0.0.1:37012 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=845 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=369 (was 340) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=3204 (was 3308) 2023-02-07 16:54:15,377 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=845, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=174, AvailableMemoryMB=3203 2023-02-07 16:54:15,378 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:15,378 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,378 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,424 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:54:15,424 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:54:15,426 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:15,428 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:15,429 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:15,431 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,432 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d empty. 2023-02-07 16:54:15,432 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,433 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-07 16:54:15,445 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:15,446 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => ec12db53da84ded9dec5333b54c85b5d, NAME => 'testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:15,454 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:15,454 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing ec12db53da84ded9dec5333b54c85b5d, disabling compactions & flushes 2023-02-07 16:54:15,455 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,455 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,455 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. after waiting 0 ms 2023-02-07 16:54:15,455 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,455 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,455 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for ec12db53da84ded9dec5333b54c85b5d: Waiting for close lock at 1675788855454Disabling compacts and flushes for region at 1675788855454Disabling writes for close at 1675788855455 (+1 ms)Writing region close event to WAL at 1675788855455Closed at 1675788855455 2023-02-07 16:54:15,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:15,458 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788855458"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788855458"}]},"ts":"1675788855458"} 2023-02-07 16:54:15,460 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:15,461 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:15,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788855461"}]},"ts":"1675788855461"} 2023-02-07 16:54:15,463 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-07 16:54:15,496 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:15,498 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:15,498 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:15,498 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:15,498 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:15,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, ASSIGN}] 2023-02-07 16:54:15,500 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, ASSIGN 2023-02-07 16:54:15,501 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:15,652 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:15,653 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=ec12db53da84ded9dec5333b54c85b5d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:15,653 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788855653"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788855653"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788855653"}]},"ts":"1675788855653"} 2023-02-07 16:54:15,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:15,808 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:15,809 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 16:54:15,811 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:48242, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 16:54:15,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ec12db53da84ded9dec5333b54c85b5d, NAME => 'testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:15,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:15,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:15,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,818 INFO [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,820 DEBUG [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1 2023-02-07 16:54:15,820 DEBUG [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1 2023-02-07 16:54:15,820 INFO [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec12db53da84ded9dec5333b54c85b5d columnFamilyName cf1 2023-02-07 16:54:15,821 INFO [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] regionserver.HStore(310): Store=ec12db53da84ded9dec5333b54c85b5d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:15,821 INFO [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,822 DEBUG [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2 2023-02-07 16:54:15,823 DEBUG [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2 2023-02-07 16:54:15,823 INFO [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec12db53da84ded9dec5333b54c85b5d columnFamilyName cf2 2023-02-07 16:54:15,824 INFO [StoreOpener-ec12db53da84ded9dec5333b54c85b5d-1] regionserver.HStore(310): Store=ec12db53da84ded9dec5333b54c85b5d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:15,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:15,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:15,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:15,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened ec12db53da84ded9dec5333b54c85b5d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75322115, jitterRate=0.12238697707653046}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:15,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for ec12db53da84ded9dec5333b54c85b5d: Running coprocessor pre-open hook at 1675788855816Writing region info on filesystem at 1675788855816Initializing all the Stores at 1675788855817 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788855818 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788855818Cleaning up temporary data from old regions at 1675788855827 (+9 ms)Cleaning up detritus from prior splits at 1675788855827Running coprocessor post-open hooks at 1675788855833 (+6 ms)Region opened successfully at 1675788855833 2023-02-07 16:54:15,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d., pid=27, masterSystemTime=1675788855808 2023-02-07 16:54:15,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,840 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=ec12db53da84ded9dec5333b54c85b5d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:15,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788855840"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788855840"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788855840"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788855840"}]},"ts":"1675788855840"} 2023-02-07 16:54:15,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-07 16:54:15,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 in 188 msec 2023-02-07 16:54:15,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-07 16:54:15,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, ASSIGN in 348 msec 2023-02-07 16:54:15,850 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:15,850 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788855850"}]},"ts":"1675788855850"} 2023-02-07 16:54:15,852 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-07 16:54:15,897 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:15,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 472 msec 2023-02-07 16:54:15,930 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 16:54:15,934 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:48258, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 16:54:15,941 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:15,941 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:15,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:15,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 16:54:15,945 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 16:54:15,945 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:15,946 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 16:54:15,946 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 16:54:15,946 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:15,946 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,959 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:15,959 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:15,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:15,959 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:15,959 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:15,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:15,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:15,959 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:15,960 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:15,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:15,961 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:15,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:15,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:15,962 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:15,962 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:15,962 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:15,962 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:15,962 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:15,963 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:15,963 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. started... 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:15,963 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:15,964 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:54:15,970 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:15,975 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/26a75cd03b9b45f68b39d99344a18754 2023-02-07 16:54:15,980 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:15,980 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:15,980 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:15,980 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:15,980 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:15,993 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/8e69a04aefa647e690168ba294b9640b 2023-02-07 16:54:15,998 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/26a75cd03b9b45f68b39d99344a18754 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 2023-02-07 16:54:16,004 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:16,005 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/8e69a04aefa647e690168ba294b9640b as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b 2023-02-07 16:54:16,011 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:16,012 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 48ms, sequenceid=6, compaction requested=false 2023-02-07 16:54:16,012 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-07 16:54:16,012 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec12db53da84ded9dec5333b54c85b5d: 2023-02-07 16:54:16,012 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:16,012 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:16,012 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:16,012 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,012 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,013 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,022 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,022 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,023 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,023 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 16:54:16,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a47c93b[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:16,025 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 16:54:16,025 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,033 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,034 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:16,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:16,043 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,043 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,043 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,044 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,043 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,044 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,054 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,054 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,054 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,054 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:54:16,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 16:54:16,059 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 16:54:16,059 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:16,059 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:16,059 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 16:54:16,059 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:16,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,065 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,066 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,068 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,068 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,068 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 16:54:16,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,075 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,076 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:16,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,076 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,076 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:16,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,177 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:16,177 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:16,177 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:16,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:16,198 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:16,198 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:16,199 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:16,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:16,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 16:54:16,201 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 16:54:16,201 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,202 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 16:54:16,202 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 16:54:16,202 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,202 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:16,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,214 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,214 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,215 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,215 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,215 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,215 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,216 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,216 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. started... 2023-02-07 16:54:16,216 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,217 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,217 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:16,222 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,232 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/c2623134f4144cbfa570357be7be979c 2023-02-07 16:54:16,233 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,233 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,233 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,233 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,233 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,249 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/392ce49e838d4a5c816e2fa7ddd65a2a 2023-02-07 16:54:16,256 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/c2623134f4144cbfa570357be7be979c as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c 2023-02-07 16:54:16,262 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:16,263 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/392ce49e838d4a5c816e2fa7ddd65a2a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a 2023-02-07 16:54:16,269 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:16,270 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 53ms, sequenceid=19, compaction requested=false 2023-02-07 16:54:16,270 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec12db53da84ded9dec5333b54c85b5d: 2023-02-07 16:54:16,270 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:16,270 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:16,270 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:16,270 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,270 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,270 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,275 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,276 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,276 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 16:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bce0e31[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:16,278 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 16:54:16,278 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,285 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,285 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,285 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,286 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,286 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:16,320 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,320 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,320 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,320 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,338 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,349 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,349 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,349 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 16:54:16,353 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 16:54:16,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:16,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:16,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 16:54:16,353 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,413 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,414 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,414 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,416 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,416 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,417 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,503 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 16:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,503 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,503 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:16,503 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,504 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,505 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,505 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,505 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,505 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,505 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:16,505 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:16,605 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:16,605 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:16,605 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:16,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:16,627 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:16,627 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:16,628 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:16,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:16,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 16:54:16,630 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 16:54:16,630 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,631 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 16:54:16,631 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 16:54:16,631 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,631 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,654 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,654 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,654 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,655 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,656 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,656 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,656 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,656 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,658 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,658 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:16,659 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:16,659 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,659 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,659 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:16,659 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,659 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. started... 2023-02-07 16:54:16,659 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:16,664 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,673 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/f63a482123e24d4bb9bccafefbeb450c 2023-02-07 16:54:16,675 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,675 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,675 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,675 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,675 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,692 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/b8ef5806913945f4801b057b799ab3f1 2023-02-07 16:54:16,698 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/f63a482123e24d4bb9bccafefbeb450c as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c 2023-02-07 16:54:16,704 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:16,705 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/b8ef5806913945f4801b057b799ab3f1 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 2023-02-07 16:54:16,712 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:16,713 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 54ms, sequenceid=32, compaction requested=true 2023-02-07 16:54:16,713 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec12db53da84ded9dec5333b54c85b5d: 2023-02-07 16:54:16,713 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:16,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:16,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:16,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,717 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,717 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,717 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,718 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 16:54:16,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fed1a42[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:16,720 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 16:54:16,720 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,728 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:16,728 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:16,729 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:16,738 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,738 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,738 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,749 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:16,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,749 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,750 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,750 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:16,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:54:16,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 16:54:16,754 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 16:54:16,754 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:16,754 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:16,754 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 16:54:16,754 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:16,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,759 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,760 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,760 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:16,760 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,761 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:16,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,762 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:16,763 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:16,763 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,770 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:16,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 16:54:16,771 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:16,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:16,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:16,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,772 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,872 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:16,872 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:16,872 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:16,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:16,895 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:16,895 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:16,896 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:16,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:16,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 16:54:16,898 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 16:54:16,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,899 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 16:54:16,899 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 16:54:16,900 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,900 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,952 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,952 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,952 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,952 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:16,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:16,953 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:16,953 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:16,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:16,954 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:16,956 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,956 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:16,956 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,956 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,957 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,956 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:16,958 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,959 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:16,959 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,960 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,959 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:16,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:16,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:16,960 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:16,960 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:16,960 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. started... 2023-02-07 16:54:16,960 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:16,973 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/394aa88e3e0a4698b963a0ab72496628 2023-02-07 16:54:16,980 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,987 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/80031a970b694a9cb88ad256e0ced2f7 2023-02-07 16:54:16,991 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,991 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,991 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,992 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:16,992 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:16,993 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/394aa88e3e0a4698b963a0ab72496628 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 2023-02-07 16:54:17,000 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:17,001 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/80031a970b694a9cb88ad256e0ced2f7 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 2023-02-07 16:54:17,008 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:17,009 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 49ms, sequenceid=45, compaction requested=true 2023-02-07 16:54:17,009 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec12db53da84ded9dec5333b54c85b5d: 2023-02-07 16:54:17,009 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:17,009 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:17,009 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:17,009 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,009 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,009 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,022 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,022 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,023 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,023 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 16:54:17,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ed0a04c[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:17,025 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 16:54:17,025 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,033 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:17,033 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,034 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:17,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:17,043 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,043 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,043 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,044 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,055 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,054 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,055 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:54:17,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 16:54:17,060 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 16:54:17,060 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:17,061 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:17,061 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 16:54:17,061 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:17,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,065 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,065 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,065 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,065 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,065 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,067 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,068 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,068 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,075 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 16:54:17,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:17,075 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,076 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,076 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:17,076 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,077 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,177 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:17,177 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:17,177 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:17,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:17,200 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:17,200 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:17,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:17,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:17,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 16:54:17,204 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 16:54:17,204 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,204 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 16:54:17,204 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 16:54:17,205 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,205 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:17,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:17,213 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:17,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:17,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:17,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,214 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:17,217 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:17,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:17,217 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. started... 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,217 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:17,217 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,223 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,231 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/78f4097736934275849f637a98bbb970 2023-02-07 16:54:17,233 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,233 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,233 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,233 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,233 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,246 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 2023-02-07 16:54:17,252 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/78f4097736934275849f637a98bbb970 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 2023-02-07 16:54:17,257 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:17,258 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 2023-02-07 16:54:17,263 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:17,265 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 48ms, sequenceid=58, compaction requested=true 2023-02-07 16:54:17,265 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec12db53da84ded9dec5333b54c85b5d: 2023-02-07 16:54:17,265 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:17,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:17,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:17,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,265 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,275 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,275 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,276 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 16:54:17,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a8ab2bc[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:17,278 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 16:54:17,278 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,285 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:17,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:17,286 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,286 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:17,286 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,287 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,287 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:17,296 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,296 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,296 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,296 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,307 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,307 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,307 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,317 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,317 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,317 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:54:17,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 16:54:17,321 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 16:54:17,321 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:17,321 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:17,321 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 16:54:17,321 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:17,353 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,353 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,353 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,353 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,354 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,353 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,353 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,354 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,354 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,355 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,356 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,356 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,356 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,370 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 16:54:17,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,371 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:17,370 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,371 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:17,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,371 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,371 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,371 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,371 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,372 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,372 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,372 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,372 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,372 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,471 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:17,471 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:17,471 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:17,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:17,496 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:17,496 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:17,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:17,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:17,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-07 16:54:17,500 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-07 16:54:17,501 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,501 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-07 16:54:17,501 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-07 16:54:17,501 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,501 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,506 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,507 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,506 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,506 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,507 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,507 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-07 16:54:17,508 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:17,508 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:17,508 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:17,508 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:17,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-07 16:54:17,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:17,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:17,509 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:17,509 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:17,509 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. started... 2023-02-07 16:54:17,510 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:17,517 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,527 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,528 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,528 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,528 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/b35c64bd62af469c8ffabaffffbfdbfb 2023-02-07 16:54:17,528 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,528 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,548 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/8b75b6bc8f2a49b897261a717f72c0da 2023-02-07 16:54:17,553 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/b35c64bd62af469c8ffabaffffbfdbfb as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb 2023-02-07 16:54:17,559 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:17,560 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/8b75b6bc8f2a49b897261a717f72c0da as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da 2023-02-07 16:54:17,566 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:17,567 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 58ms, sequenceid=71, compaction requested=true 2023-02-07 16:54:17,567 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ec12db53da84ded9dec5333b54c85b5d: 2023-02-07 16:54:17,567 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:17,567 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:17,567 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:17,567 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,568 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-07 16:54:17,568 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-07 16:54:17,580 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,580 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,581 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,581 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:17,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-07 16:54:17,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@313e8c5a[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:17,583 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-07 16:54:17,583 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:17,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,591 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,591 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:17,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,591 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,592 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-07 16:54:17,592 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,592 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:17,592 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-07 16:54:17,592 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:17,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,601 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,602 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,612 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,612 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,612 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,622 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-07 16:54:17,623 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,623 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-07 16:54:17,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:54:17,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-07 16:54:17,626 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-07 16:54:17,626 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:17,626 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:17,626 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-07 16:54:17,626 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:17,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:17,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:17,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,634 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,634 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:17,635 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:17,636 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-07 16:54:17,637 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,637 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,643 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitWithoutPONR because node does not exist (not an error) 2023-02-07 16:54:17,643 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,643 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,644 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-07 16:54:17,644 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:17,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,644 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:17,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:17,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:17,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-07 16:54:17,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:17,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:17,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-07 16:54:17,645 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-07 16:54:17,745 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:17,745 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:17,745 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:17,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:18,011 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-07 16:54:18,012 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:18,013 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,014 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,015 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,015 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:18,016 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:18,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:18,264 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,265 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:18,265 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:18,265 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:18,267 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-07 16:54:18,268 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:18,268 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:18,268 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:18,268 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:18,269 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:18,273 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-07 16:54:18,275 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:18,276 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:18,276 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:18,276 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:18,277 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,277 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:18,278 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,278 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,279 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:18,279 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,280 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:18,280 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,280 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,280 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,280 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-07 16:54:18,281 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:18,282 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:18,283 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788888283 2023-02-07 16:54:18,285 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,285 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:18,289 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:18,289 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:18,290 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:18,290 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788918290 2023-02-07 16:54:18,290 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788978290 2023-02-07 16:54:18,290 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:54:18,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:18,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:18,292 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:18,293 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:18,540 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,540 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:18,540 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:18,540 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:18,541 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:18,541 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:18,541 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:18,541 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:18,541 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:18,542 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:18,544 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 16:54:18,545 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:18,545 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:18,545 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:18,546 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:18,546 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,547 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:18,547 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,547 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,547 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:18,548 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,548 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:18,548 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,548 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,548 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,549 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:18,549 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:18,550 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:18,550 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788888550 2023-02-07 16:54:18,551 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,551 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:18,555 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:18,556 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:18,556 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:18,556 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788918556 2023-02-07 16:54:18,556 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788978556 2023-02-07 16:54:18,556 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:54:18,556 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:18,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN}] 2023-02-07 16:54:18,559 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:18,559 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:18,559 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:18,806 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,806 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:18,806 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:18,807 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:18,807 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:18,807 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:18,807 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:18,807 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:18,807 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:18,808 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:18,809 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 16:54:18,810 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:18,810 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:18,810 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:18,811 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:18,811 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,811 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:18,812 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,812 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:18,812 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:18,812 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,812 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:18,812 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,813 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,813 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,813 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:18,814 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:18,814 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:18,815 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788888815 2023-02-07 16:54:18,816 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:18,816 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:18,819 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:18,819 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:18,819 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:18,819 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788918819 2023-02-07 16:54:18,819 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788978819 2023-02-07 16:54:18,819 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:18,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN}] 2023-02-07 16:54:18,820 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:18,822 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:18,823 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:18,823 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788858823"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788858823"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788858823"}]},"ts":"1675788858823"} 2023-02-07 16:54:18,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:18,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:18,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:18,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:18,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:19,070 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,070 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:19,070 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:19,070 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:19,070 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:19,070 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:19,071 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:19,071 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:19,071 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:19,072 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:19,073 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:19,073 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 16:54:19,073 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:19,074 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:19,074 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:19,074 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:19,075 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,075 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:19,075 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,075 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,076 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:19,076 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,077 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,077 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:19,077 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,077 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,077 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,077 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:19,077 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,077 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:19,078 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:19,079 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:19,079 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d to restore RIT 2023-02-07 16:54:19,079 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788889079 2023-02-07 16:54:19,080 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,080 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:19,083 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:19,083 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:19,083 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:19,083 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788919083 2023-02-07 16:54:19,083 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788979083 2023-02-07 16:54:19,084 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:19,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,084 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788859084"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788859084"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788859084"}]},"ts":"1675788859084"} 2023-02-07 16:54:19,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:19,086 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:19,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing ec12db53da84ded9dec5333b54c85b5d, disabling compactions & flushes 2023-02-07 16:54:19,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:19,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:19,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. after waiting 0 ms 2023-02-07 16:54:19,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:19,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing ec12db53da84ded9dec5333b54c85b5d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:54:19,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/12102fb4f2464dd88a2556407498cba5 2023-02-07 16:54:19,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/dca073f34f744bf0b726ae9440d4a343 2023-02-07 16:54:19,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf1/12102fb4f2464dd88a2556407498cba5 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 2023-02-07 16:54:19,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:54:19,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/.tmp/cf2/dca073f34f744bf0b726ae9440d4a343 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 2023-02-07 16:54:19,287 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:54:19,287 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 47ms, sequenceid=82, compaction requested=true 2023-02-07 16:54:19,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:54:19,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:19,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. 2023-02-07 16:54:19,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for ec12db53da84ded9dec5333b54c85b5d: Waiting for close lock at 1675788859240Running coprocessor pre-close hooks at 1675788859240Disabling compacts and flushes for region at 1675788859240Disabling writes for close at 1675788859240Obtaining lock to block concurrent updates at 1675788859240Preparing flush snapshotting stores in ec12db53da84ded9dec5333b54c85b5d at 1675788859240Finished memstore snapshotting testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675788859240Flushing stores of testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. at 1675788859241 (+1 ms)Flushing ec12db53da84ded9dec5333b54c85b5d/cf1: creating writer at 1675788859242 (+1 ms)Flushing ec12db53da84ded9dec5333b54c85b5d/cf1: appending metadata at 1675788859244 (+2 ms)Flushing ec12db53da84ded9dec5333b54c85b5d/cf1: closing flushed file at 1675788859244Flushing ec12db53da84ded9dec5333b54c85b5d/cf2: creating writer at 1675788859258 (+14 ms)Flushing ec12db53da84ded9dec5333b54c85b5d/cf2: appending metadata at 1675788859260 (+2 ms)Flushing ec12db53da84ded9dec5333b54c85b5d/cf2: closing flushed file at 1675788859260Flushing ec12db53da84ded9dec5333b54c85b5d/cf1: reopening flushed file at 1675788859275 (+15 ms)Flushing ec12db53da84ded9dec5333b54c85b5d/cf2: reopening flushed file at 1675788859281 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ec12db53da84ded9dec5333b54c85b5d in 47ms, sequenceid=82, compaction requested=true at 1675788859288 (+7 ms)Writing region close event to WAL at 1675788859305 (+17 ms)Running coprocessor post-close hooks at 1675788859309 (+4 ms)Closed at 1675788859309 2023-02-07 16:54:19,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,313 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED 2023-02-07 16:54:19,313 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788859313"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788859313"}]},"ts":"1675788859313"} 2023-02-07 16:54:19,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:19,315 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:19,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:19,324 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:54:19,334 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,334 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:19,334 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:19,334 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:19,334 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:19,335 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:19,335 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:19,335 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:19,335 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:19,336 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:19,337 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:19,337 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 16:54:19,337 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-07 16:54:19,337 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:19,338 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:19,338 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:19,338 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:19,339 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,339 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:19,339 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,339 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,340 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:19,340 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,340 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,341 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,341 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:19,341 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,341 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,341 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,341 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:19,341 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,341 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:19,341 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:19,342 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:19,343 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:19,343 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d to restore RIT 2023-02-07 16:54:19,345 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788889345 2023-02-07 16:54:19,346 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,347 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:19,349 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:19,350 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:19,350 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:19,350 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788919350 2023-02-07 16:54:19,350 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788979350 2023-02-07 16:54:19,350 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:19,350 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788859350"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788859350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788859350"}]},"ts":"1675788859350"} 2023-02-07 16:54:19,352 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:19,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-07 16:54:19,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 in 266 msec 2023-02-07 16:54:19,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:19,356 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:19,357 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:19,600 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:19,601 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:19,601 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:19,602 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:19,604 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:19,604 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 16:54:19,604 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:19,604 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:19,605 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:19,605 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:19,605 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:19,605 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,606 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:19,606 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,606 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,607 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:19,607 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,607 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,607 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,608 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:19,608 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,608 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,608 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,608 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:19,608 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:19,608 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:19,609 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:19,609 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:19,609 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d to restore RIT 2023-02-07 16:54:19,610 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788889610 2023-02-07 16:54:19,610 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,611 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:19,613 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:19,613 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:19,613 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:19,613 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788919613 2023-02-07 16:54:19,614 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788979614 2023-02-07 16:54:19,614 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:19,614 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,616 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:19,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-07 16:54:19,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN in 796 msec 2023-02-07 16:54:19,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:19,623 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:19,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:19,864 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,864 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:19,864 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:19,864 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:19,864 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:19,865 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:19,865 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:19,865 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:19,865 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:19,866 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:19,867 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:19,867 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:19,867 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:19,867 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:19,868 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:19,868 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:19,868 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:19,868 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,869 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:19,869 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,869 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:19,869 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:19,870 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,870 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:19,870 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:19,870 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:19,870 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:19,870 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:19,870 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:19,871 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:19,872 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:19,872 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788889872 2023-02-07 16:54:19,874 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,874 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:19,876 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:19,877 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:19,877 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:19,877 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788919877 2023-02-07 16:54:19,877 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788979877 2023-02-07 16:54:19,877 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:19,877 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,878 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,878 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:19,883 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:19,894 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=ec12db53da84ded9dec5333b54c85b5d, threads=14 2023-02-07 16:54:19,895 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,895 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,895 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,896 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,897 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,897 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:19,897 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,077 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,081 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,100 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,119 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,120 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,120 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,137 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,147 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,149 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,153 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,153 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,154 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,157 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,157 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,157 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region ec12db53da84ded9dec5333b54c85b5d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:54:20,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:20,197 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:20,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:20,378 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:20,378 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:20,379 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:20,380 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 16:54:20,380 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:20,380 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:20,381 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:20,381 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:20,381 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:20,382 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:20,382 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:20,382 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:20,383 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:20,383 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:20,383 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:20,384 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,384 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:20,384 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:20,384 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:20,384 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,384 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,384 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,384 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:20,384 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:20,385 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:20,385 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:20,386 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:20,386 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788890386 2023-02-07 16:54:20,388 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:20,388 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:20,390 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:20,391 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:20,391 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:20,391 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788920391 2023-02-07 16:54:20,391 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788980391 2023-02-07 16:54:20,391 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:20,401 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=ec12db53da84ded9dec5333b54c85b5d, threads=14 2023-02-07 16:54:20,402 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,402 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,402 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,402 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,402 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,402 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,403 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,403 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,403 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,403 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,403 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,404 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,404 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,404 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,549 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,565 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,585 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,588 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,594 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,625 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,626 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,626 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,641 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,644 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,648 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,653 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,653 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,656 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 for region: ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:20,656 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region ec12db53da84ded9dec5333b54c85b5d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:54:21,118 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:21,124 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:54:21,126 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:54:21,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:21,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:21,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:21,142 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,142 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:21,142 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:21,142 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:21,142 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:21,142 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:21,143 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:21,143 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:21,143 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:21,143 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:21,145 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 16:54:21,145 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:21,145 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:21,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:21,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:21,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:21,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:21,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:21,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:21,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:21,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:21,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:21,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:21,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:21,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:21,150 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,150 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,150 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,150 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:21,150 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:21,150 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:21,151 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:21,151 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:21,152 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788891152 2023-02-07 16:54:21,153 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,153 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:21,155 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:21,156 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:21,156 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:21,156 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788921156 2023-02-07 16:54:21,156 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788981156 2023-02-07 16:54:21,156 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:21,159 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:21,161 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:21,161 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:21,161 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:21,406 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:21,406 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:21,407 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:21,407 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:21,407 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:21,407 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:21,407 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:21,407 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:21,407 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:21,408 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:21,409 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 16:54:21,409 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:21,409 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:21,409 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:21,410 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:21,410 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:21,410 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:21,410 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:21,410 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:21,411 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:21,411 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:21,411 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:21,411 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,411 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN 2023-02-07 16:54:21,412 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:21,412 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:21,412 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,412 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,412 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,412 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=ec12db53da84ded9dec5333b54c85b5d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:21,412 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure ec12db53da84ded9dec5333b54c85b5d, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:21,412 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 16:54:21,413 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:21,413 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:21,413 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788891413 2023-02-07 16:54:21,415 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,415 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:21,417 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:21,417 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ec12db53da84ded9dec5333b54c85b5d, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:54:21,417 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:21,417 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788921417 2023-02-07 16:54:21,417 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788981417 2023-02-07 16:54:21,417 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:21,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788861421"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675788861421"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675788861421"}]},"ts":"1675788861421"} 2023-02-07 16:54:21,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788861421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788861421"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788861421"}]},"ts":"1675788861421"} 2023-02-07 16:54:21,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788861421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788861421"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788861421"}]},"ts":"1675788861421"} 2023-02-07 16:54:21,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, ASSIGN}] 2023-02-07 16:54:21,429 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, ASSIGN 2023-02-07 16:54:21,430 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:21,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, ASSIGN 2023-02-07 16:54:21,432 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:21,581 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:54:21,582 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=91836b86732502996543a428936bdda0, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:21,583 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788861582"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788861582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788861582"}]},"ts":"1675788861582"} 2023-02-07 16:54:21,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 91836b86732502996543a428936bdda0, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:21,588 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=4b2f65dee3751ab595b2dd0636c28158, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:21,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788861588"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788861588"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788861588"}]},"ts":"1675788861588"} 2023-02-07 16:54:21,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 4b2f65dee3751ab595b2dd0636c28158, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:21,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 91836b86732502996543a428936bdda0, NAME => 'testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 16:54:21,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:21,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 91836b86732502996543a428936bdda0 2023-02-07 16:54:21,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:21,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 91836b86732502996543a428936bdda0 2023-02-07 16:54:21,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 91836b86732502996543a428936bdda0 2023-02-07 16:54:21,743 INFO [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 91836b86732502996543a428936bdda0 2023-02-07 16:54:21,744 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1 2023-02-07 16:54:21,744 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1 2023-02-07 16:54:21,744 INFO [StoreOpener-91836b86732502996543a428936bdda0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91836b86732502996543a428936bdda0 columnFamilyName cf1 2023-02-07 16:54:21,763 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5-bottom 2023-02-07 16:54:21,768 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754-bottom 2023-02-07 16:54:21,775 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628-bottom 2023-02-07 16:54:21,782 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970-bottom 2023-02-07 16:54:21,789 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb-bottom 2023-02-07 16:54:21,796 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c-bottom 2023-02-07 16:54:21,802 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c-bottom 2023-02-07 16:54:21,802 INFO [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(310): Store=91836b86732502996543a428936bdda0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:21,802 INFO [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 91836b86732502996543a428936bdda0 2023-02-07 16:54:21,803 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2 2023-02-07 16:54:21,803 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2 2023-02-07 16:54:21,804 INFO [StoreOpener-91836b86732502996543a428936bdda0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91836b86732502996543a428936bdda0 columnFamilyName cf2 2023-02-07 16:54:21,820 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a-bottom 2023-02-07 16:54:21,825 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7-bottom 2023-02-07 16:54:21,830 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da-bottom 2023-02-07 16:54:21,835 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b-bottom 2023-02-07 16:54:21,841 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897-bottom 2023-02-07 16:54:21,847 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1-bottom 2023-02-07 16:54:21,853 DEBUG [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343-bottom 2023-02-07 16:54:21,853 INFO [StoreOpener-91836b86732502996543a428936bdda0-1] regionserver.HStore(310): Store=91836b86732502996543a428936bdda0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:21,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0 2023-02-07 16:54:21,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0 2023-02-07 16:54:21,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:21,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 91836b86732502996543a428936bdda0 2023-02-07 16:54:21,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 91836b86732502996543a428936bdda0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60607608, jitterRate=-0.09687626361846924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:21,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 91836b86732502996543a428936bdda0: Running coprocessor pre-open hook at 1675788861741Writing region info on filesystem at 1675788861741Initializing all the Stores at 1675788861742 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788861742Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788861742Cleaning up temporary data from old regions at 1675788861856 (+114 ms)Cleaning up detritus from prior splits at 1675788861856Running coprocessor post-open hooks at 1675788861859 (+3 ms)Region opened successfully at 1675788861859 2023-02-07 16:54:21,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0., pid=33, masterSystemTime=1675788861737 2023-02-07 16:54:21,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:21,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:21,861 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:21,861 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:21,861 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,862 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,862 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1912): 91836b86732502996543a428936bdda0/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:21,862 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1912): 91836b86732502996543a428936bdda0/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:21,862 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 91836b86732502996543a428936bdda0/cf2 in testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,862 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 91836b86732502996543a428936bdda0/cf1 in testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,862 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/.tmp, totalSize=34.9 K 2023-02-07 16:54:21,862 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/.tmp, totalSize=34.9 K 2023-02-07 16:54:21,863 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788855935 2023-02-07 16:54:21,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,863 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788855935 2023-02-07 16:54:21,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:21,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,864 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=91836b86732502996543a428936bdda0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:21,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4b2f65dee3751ab595b2dd0636c28158, NAME => 'testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 16:54:21,864 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788856179 2023-02-07 16:54:21,864 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788861864"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788861864"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788861864"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788861864"}]},"ts":"1675788861864"} 2023-02-07 16:54:21,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:21,864 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788856608 2023-02-07 16:54:21,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:21,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,865 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788856179 2023-02-07 16:54:21,865 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788856874 2023-02-07 16:54:21,866 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788856608 2023-02-07 16:54:21,866 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788857179 2023-02-07 16:54:21,866 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788856874 2023-02-07 16:54:21,866 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788857474 2023-02-07 16:54:21,866 INFO [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,867 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788857179 2023-02-07 16:54:21,867 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788857747 2023-02-07 16:54:21,867 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788857474 2023-02-07 16:54:21,867 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1 2023-02-07 16:54:21,867 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1 2023-02-07 16:54:21,868 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788857747 2023-02-07 16:54:21,869 INFO [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b2f65dee3751ab595b2dd0636c28158 columnFamilyName cf1 2023-02-07 16:54:21,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-07 16:54:21,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 91836b86732502996543a428936bdda0, server=jenkins-hbase12.apache.org,41937,1675788846347 in 281 msec 2023-02-07 16:54:21,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, ASSIGN in 444 msec 2023-02-07 16:54:21,883 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] throttle.PressureAwareThroughputController(145): 91836b86732502996543a428936bdda0#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:21,887 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] throttle.PressureAwareThroughputController(145): 91836b86732502996543a428936bdda0#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:21,890 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5-top 2023-02-07 16:54:21,905 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754-top 2023-02-07 16:54:21,912 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628-top 2023-02-07 16:54:21,916 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/.tmp/cf2/622b82381e934095886bbce60553c5ed as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/622b82381e934095886bbce60553c5ed 2023-02-07 16:54:21,917 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/.tmp/cf1/ddb4f53ab7ce4f6ba936b3fa02cf6a58 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/ddb4f53ab7ce4f6ba936b3fa02cf6a58 2023-02-07 16:54:21,921 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970-top 2023-02-07 16:54:21,923 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 91836b86732502996543a428936bdda0/cf2 of 91836b86732502996543a428936bdda0 into 622b82381e934095886bbce60553c5ed(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:21,923 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 91836b86732502996543a428936bdda0: 2023-02-07 16:54:21,923 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0., storeName=91836b86732502996543a428936bdda0/cf2, priority=9, startTime=1675788861860; duration=0sec 2023-02-07 16:54:21,924 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:21,925 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb-top 2023-02-07 16:54:21,926 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 91836b86732502996543a428936bdda0/cf1 of 91836b86732502996543a428936bdda0 into ddb4f53ab7ce4f6ba936b3fa02cf6a58(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:21,926 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 91836b86732502996543a428936bdda0: 2023-02-07 16:54:21,926 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0., storeName=91836b86732502996543a428936bdda0/cf1, priority=9, startTime=1675788861860; duration=0sec 2023-02-07 16:54:21,927 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:21,931 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c-top 2023-02-07 16:54:21,937 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c-top 2023-02-07 16:54:21,937 INFO [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(310): Store=4b2f65dee3751ab595b2dd0636c28158/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:21,937 INFO [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,938 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2 2023-02-07 16:54:21,938 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2 2023-02-07 16:54:21,939 INFO [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b2f65dee3751ab595b2dd0636c28158 columnFamilyName cf2 2023-02-07 16:54:21,955 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a-top 2023-02-07 16:54:21,959 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7-top 2023-02-07 16:54:21,963 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da-top 2023-02-07 16:54:21,967 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b-top 2023-02-07 16:54:21,974 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897-top 2023-02-07 16:54:21,979 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1-top 2023-02-07 16:54:21,984 DEBUG [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343-top 2023-02-07 16:54:21,984 INFO [StoreOpener-4b2f65dee3751ab595b2dd0636c28158-1] regionserver.HStore(310): Store=4b2f65dee3751ab595b2dd0636c28158/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:21,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:21,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:21,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 4b2f65dee3751ab595b2dd0636c28158; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63851781, jitterRate=-0.048534318804740906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:21,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 4b2f65dee3751ab595b2dd0636c28158: Running coprocessor pre-open hook at 1675788861865Writing region info on filesystem at 1675788861865Initializing all the Stores at 1675788861866 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788861866Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788861866Cleaning up temporary data from old regions at 1675788861986 (+120 ms)Cleaning up detritus from prior splits at 1675788861986Running coprocessor post-open hooks at 1675788861989 (+3 ms)Region opened successfully at 1675788861989 2023-02-07 16:54:21,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158., pid=34, masterSystemTime=1675788861737 2023-02-07 16:54:21,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:21,990 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:21,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:21,990 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:21,991 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,991 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1912): 4b2f65dee3751ab595b2dd0636c28158/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:21,991 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,991 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1912): 4b2f65dee3751ab595b2dd0636c28158/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:21,991 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4b2f65dee3751ab595b2dd0636c28158/cf1 in testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,991 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4b2f65dee3751ab595b2dd0636c28158/cf2 in testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,991 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5-top] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/.tmp, totalSize=34.9 K 2023-02-07 16:54:21,991 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343-top] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/.tmp, totalSize=34.9 K 2023-02-07 16:54:21,992 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788855935 2023-02-07 16:54:21,992 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788855935 2023-02-07 16:54:21,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,992 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788856179 2023-02-07 16:54:21,992 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788856179 2023-02-07 16:54:21,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:21,993 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788856608 2023-02-07 16:54:21,993 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788856608 2023-02-07 16:54:21,993 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=4b2f65dee3751ab595b2dd0636c28158, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:21,993 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788861993"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788861993"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788861993"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788861993"}]},"ts":"1675788861993"} 2023-02-07 16:54:21,993 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788856874 2023-02-07 16:54:21,993 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788856874 2023-02-07 16:54:21,993 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675788857179 2023-02-07 16:54:21,993 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675788857179 2023-02-07 16:54:21,994 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675788857474 2023-02-07 16:54:21,994 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675788857474 2023-02-07 16:54:21,994 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788857747 2023-02-07 16:54:21,994 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788857747 2023-02-07 16:54:21,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-07 16:54:21,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 4b2f65dee3751ab595b2dd0636c28158, server=jenkins-hbase12.apache.org,41937,1675788846347 in 405 msec 2023-02-07 16:54:22,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-07 16:54:22,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, ASSIGN in 572 msec 2023-02-07 16:54:22,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 in 3.9880 sec 2023-02-07 16:54:22,009 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] throttle.PressureAwareThroughputController(145): 4b2f65dee3751ab595b2dd0636c28158#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:22,014 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4b2f65dee3751ab595b2dd0636c28158#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:22,036 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/.tmp/cf2/1a6f24679f104029b8c957bf19d2a475 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/1a6f24679f104029b8c957bf19d2a475 2023-02-07 16:54:22,038 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/.tmp/cf1/93e402ca9f344117b2b11d778e752ef6 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/93e402ca9f344117b2b11d778e752ef6 2023-02-07 16:54:22,042 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4b2f65dee3751ab595b2dd0636c28158/cf2 of 4b2f65dee3751ab595b2dd0636c28158 into 1a6f24679f104029b8c957bf19d2a475(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:22,042 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4b2f65dee3751ab595b2dd0636c28158: 2023-02-07 16:54:22,042 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158., storeName=4b2f65dee3751ab595b2dd0636c28158/cf2, priority=9, startTime=1675788861990; duration=0sec 2023-02-07 16:54:22,042 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:22,044 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4b2f65dee3751ab595b2dd0636c28158/cf1 of 4b2f65dee3751ab595b2dd0636c28158 into 93e402ca9f344117b2b11d778e752ef6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:22,044 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4b2f65dee3751ab595b2dd0636c28158: 2023-02-07 16:54:22,044 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158., storeName=4b2f65dee3751ab595b2dd0636c28158/cf1, priority=9, startTime=1675788861990; duration=0sec 2023-02-07 16:54:22,044 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:22,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41937] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:48258 deadline: 1675788922169, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d. is not online on jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:22,283 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:54:22,293 INFO [Listener at localhost.localdomain/34435] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 16:54:22,295 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:54:22,313 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:22,313 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,313 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,315 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,315 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,316 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-07 16:54:22,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitWithoutPONR 2023-02-07 16:54:22,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:22,319 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,319 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 16:54:22,320 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788862320"}]},"ts":"1675788862320"} 2023-02-07 16:54:22,322 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-07 16:54:22,401 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-07 16:54:22,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, UNASSIGN}] 2023-02-07 16:54:22,403 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, UNASSIGN 2023-02-07 16:54:22,404 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=4b2f65dee3751ab595b2dd0636c28158, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:22,404 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788862404"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788862404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788862404"}]},"ts":"1675788862404"} 2023-02-07 16:54:22,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 4b2f65dee3751ab595b2dd0636c28158, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:22,408 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, UNASSIGN 2023-02-07 16:54:22,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=91836b86732502996543a428936bdda0, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:22,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788862409"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788862409"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788862409"}]},"ts":"1675788862409"} 2023-02-07 16:54:22,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 91836b86732502996543a428936bdda0, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:22,420 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,421 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 16:54:22,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:22,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 4b2f65dee3751ab595b2dd0636c28158, disabling compactions & flushes 2023-02-07 16:54:22,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:22,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:22,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. after waiting 0 ms 2023-02-07 16:54:22,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:22,571 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5-top] to archive 2023-02-07 16:54:22,572 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:22,574 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,576 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,577 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,579 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,580 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,582 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,584 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,585 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343-top] to archive 2023-02-07 16:54:22,586 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:22,588 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,589 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,590 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,591 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,592 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,594 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,595 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:54:22,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:22,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158. 2023-02-07 16:54:22,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 4b2f65dee3751ab595b2dd0636c28158: Waiting for close lock at 1675788862558Running coprocessor pre-close hooks at 1675788862558Disabling compacts and flushes for region at 1675788862558Disabling writes for close at 1675788862558Writing region close event to WAL at 1675788862595 (+37 ms)Running coprocessor post-close hooks at 1675788862599 (+4 ms)Closed at 1675788862599 2023-02-07 16:54:22,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:22,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 91836b86732502996543a428936bdda0 2023-02-07 16:54:22,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 91836b86732502996543a428936bdda0, disabling compactions & flushes 2023-02-07 16:54:22,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:22,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:22,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. after waiting 0 ms 2023-02-07 16:54:22,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:22,601 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=4b2f65dee3751ab595b2dd0636c28158, regionState=CLOSED 2023-02-07 16:54:22,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788862601"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788862601"}]},"ts":"1675788862601"} 2023-02-07 16:54:22,602 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5-bottom] to archive 2023-02-07 16:54:22,605 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:22,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-07 16:54:22,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 4b2f65dee3751ab595b2dd0636c28158, server=jenkins-hbase12.apache.org,41937,1675788846347 in 197 msec 2023-02-07 16:54:22,607 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4b2f65dee3751ab595b2dd0636c28158, UNASSIGN in 204 msec 2023-02-07 16:54:22,608 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,610 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,611 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,612 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,613 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,615 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,617 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343-bottom] to archive 2023-02-07 16:54:22,618 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:22,619 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,621 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,622 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,622 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,622 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 16:54:22,623 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,624 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,626 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,627 DEBUG [StoreCloser-testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:54:22,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:22,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0. 2023-02-07 16:54:22,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 91836b86732502996543a428936bdda0: Waiting for close lock at 1675788862601Running coprocessor pre-close hooks at 1675788862601Disabling compacts and flushes for region at 1675788862601Disabling writes for close at 1675788862601Writing region close event to WAL at 1675788862628 (+27 ms)Running coprocessor post-close hooks at 1675788862631 (+3 ms)Closed at 1675788862631 2023-02-07 16:54:22,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 91836b86732502996543a428936bdda0 2023-02-07 16:54:22,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=91836b86732502996543a428936bdda0, regionState=CLOSED 2023-02-07 16:54:22,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788862638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788862638"}]},"ts":"1675788862638"} 2023-02-07 16:54:22,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-07 16:54:22,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 91836b86732502996543a428936bdda0, server=jenkins-hbase12.apache.org,41937,1675788846347 in 230 msec 2023-02-07 16:54:22,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-07 16:54:22,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=91836b86732502996543a428936bdda0, UNASSIGN in 240 msec 2023-02-07 16:54:22,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788862645"}]},"ts":"1675788862645"} 2023-02-07 16:54:22,647 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-07 16:54:22,664 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-07 16:54:22,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 349 msec 2023-02-07 16:54:22,923 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,924 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-07 16:54:22,925 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-07 16:54:22,925 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,925 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitWithoutPONR 2023-02-07 16:54:22,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:22,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:22,929 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:22,929 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:22,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:22,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-07 16:54:22,935 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,935 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:22,935 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0 2023-02-07 16:54:22,937 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/recovered.edits] 2023-02-07 16:54:22,938 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/recovered.edits] 2023-02-07 16:54:22,938 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/recovered.edits] 2023-02-07 16:54:22,941 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/.regioninfo to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/.regioninfo 2023-02-07 16:54:22,941 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/12102fb4f2464dd88a2556407498cba5 2023-02-07 16:54:22,941 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/.regioninfo to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/.regioninfo 2023-02-07 16:54:22,942 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/26a75cd03b9b45f68b39d99344a18754 2023-02-07 16:54:22,944 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,944 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/12102fb4f2464dd88a2556407498cba5.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,944 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/394aa88e3e0a4698b963a0ab72496628 2023-02-07 16:54:22,945 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,945 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/26a75cd03b9b45f68b39d99344a18754.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,945 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/78f4097736934275849f637a98bbb970 2023-02-07 16:54:22,946 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,946 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/394aa88e3e0a4698b963a0ab72496628.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,946 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/b35c64bd62af469c8ffabaffffbfdbfb 2023-02-07 16:54:22,947 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,948 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/c2623134f4144cbfa570357be7be979c 2023-02-07 16:54:22,948 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/78f4097736934275849f637a98bbb970.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,949 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,949 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf1/f63a482123e24d4bb9bccafefbeb450c 2023-02-07 16:54:22,949 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/b35c64bd62af469c8ffabaffffbfdbfb.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,950 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,950 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/c2623134f4144cbfa570357be7be979c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,951 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,951 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf1/f63a482123e24d4bb9bccafefbeb450c.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,952 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/392ce49e838d4a5c816e2fa7ddd65a2a 2023-02-07 16:54:22,953 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/80031a970b694a9cb88ad256e0ced2f7 2023-02-07 16:54:22,955 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,955 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/392ce49e838d4a5c816e2fa7ddd65a2a.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,956 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8b75b6bc8f2a49b897261a717f72c0da 2023-02-07 16:54:22,956 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,956 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/80031a970b694a9cb88ad256e0ced2f7.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,957 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8e69a04aefa647e690168ba294b9640b 2023-02-07 16:54:22,958 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,958 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/8b75b6bc8f2a49b897261a717f72c0da.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,959 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/8eb31ca52cda4c3fa03842ab2b0a1897 2023-02-07 16:54:22,959 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,960 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/8e69a04aefa647e690168ba294b9640b.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,961 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,961 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/b8ef5806913945f4801b057b799ab3f1 2023-02-07 16:54:22,961 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/8eb31ca52cda4c3fa03842ab2b0a1897.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,963 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,963 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/b8ef5806913945f4801b057b799ab3f1.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,963 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/cf2/dca073f34f744bf0b726ae9440d4a343 2023-02-07 16:54:22,965 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/4b2f65dee3751ab595b2dd0636c28158/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,965 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/91836b86732502996543a428936bdda0/cf2/dca073f34f744bf0b726ae9440d4a343.ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,967 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/recovered.edits/85.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d/recovered.edits/85.seqid 2023-02-07 16:54:22,968 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/93e402ca9f344117b2b11d778e752ef6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf1/93e402ca9f344117b2b11d778e752ef6 2023-02-07 16:54:22,968 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/ddb4f53ab7ce4f6ba936b3fa02cf6a58 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf1/ddb4f53ab7ce4f6ba936b3fa02cf6a58 2023-02-07 16:54:22,968 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/ec12db53da84ded9dec5333b54c85b5d 2023-02-07 16:54:22,971 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/1a6f24679f104029b8c957bf19d2a475 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/cf2/1a6f24679f104029b8c957bf19d2a475 2023-02-07 16:54:22,971 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/622b82381e934095886bbce60553c5ed to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/cf2/622b82381e934095886bbce60553c5ed 2023-02-07 16:54:22,974 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0/recovered.edits/90.seqid 2023-02-07 16:54:22,975 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158/recovered.edits/90.seqid 2023-02-07 16:54:22,975 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/91836b86732502996543a428936bdda0 2023-02-07 16:54:22,975 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitWithoutPONR/4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:22,975 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-07 16:54:22,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:22,980 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-07 16:54:22,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-07 16:54:22,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:22,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-07 16:54:22,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788862984"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:22,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788862984"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:22,985 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788862984"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:22,987 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:54:22,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ec12db53da84ded9dec5333b54c85b5d, NAME => 'testSplitWithoutPONR,,1675788855425.ec12db53da84ded9dec5333b54c85b5d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 91836b86732502996543a428936bdda0, NAME => 'testSplitWithoutPONR,,1675788858012.91836b86732502996543a428936bdda0.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 4b2f65dee3751ab595b2dd0636c28158, NAME => 'testSplitWithoutPONR,41,1675788858012.4b2f65dee3751ab595b2dd0636c28158.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 16:54:22,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-07 16:54:22,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788862987"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:22,989 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-07 16:54:23,030 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:23,030 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:23,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-07 16:54:23,033 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:23,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 107 msec 2023-02-07 16:54:23,232 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:23,232 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:23,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-07 16:54:23,234 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-07 16:54:23,248 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=403 (was 391) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:60758 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:35136 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:60752 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:37042 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:37116 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:37146 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:37516 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:60300 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase12:41937-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:60210 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:35112 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:36962 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:60604 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:36966 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:37384 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:60618 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1919099930_15 at /127.0.0.1:50202 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=859 (was 845) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=388 (was 369) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=3073 (was 3203) 2023-02-07 16:54:23,262 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=403, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=388, ProcessCount=174, AvailableMemoryMB=3073 2023-02-07 16:54:23,262 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:23,262 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:23,263 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:23,275 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:54:23,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:54:23,277 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:23,278 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:23,279 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:23,281 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,281 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 empty. 2023-02-07 16:54:23,282 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,282 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-07 16:54:23,293 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:23,294 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => ab96ff60256f44e85e21a1807242ec57, NAME => 'testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:23,302 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:23,302 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing ab96ff60256f44e85e21a1807242ec57, disabling compactions & flushes 2023-02-07 16:54:23,302 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,302 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,302 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. after waiting 0 ms 2023-02-07 16:54:23,302 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,302 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,302 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for ab96ff60256f44e85e21a1807242ec57: Waiting for close lock at 1675788863302Disabling compacts and flushes for region at 1675788863302Disabling writes for close at 1675788863302Writing region close event to WAL at 1675788863302Closed at 1675788863302 2023-02-07 16:54:23,305 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:23,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788863305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788863305"}]},"ts":"1675788863305"} 2023-02-07 16:54:23,307 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:23,308 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:23,308 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788863308"}]},"ts":"1675788863308"} 2023-02-07 16:54:23,309 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-07 16:54:23,359 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:23,361 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:23,361 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:23,361 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:23,361 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:23,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN}] 2023-02-07 16:54:23,363 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:23,364 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=false 2023-02-07 16:54:23,514 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:23,515 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,515 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788863515"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788863515"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788863515"}]},"ts":"1675788863515"} 2023-02-07 16:54:23,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:23,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ab96ff60256f44e85e21a1807242ec57, NAME => 'testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:23,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:23,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:23,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,678 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,680 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1 2023-02-07 16:54:23,680 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1 2023-02-07 16:54:23,680 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab96ff60256f44e85e21a1807242ec57 columnFamilyName cf1 2023-02-07 16:54:23,681 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(310): Store=ab96ff60256f44e85e21a1807242ec57/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:23,681 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,682 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2 2023-02-07 16:54:23,682 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2 2023-02-07 16:54:23,683 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab96ff60256f44e85e21a1807242ec57 columnFamilyName cf2 2023-02-07 16:54:23,683 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(310): Store=ab96ff60256f44e85e21a1807242ec57/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:23,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:23,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:23,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:23,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened ab96ff60256f44e85e21a1807242ec57; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69144633, jitterRate=0.03033532202243805}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:23,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for ab96ff60256f44e85e21a1807242ec57: Running coprocessor pre-open hook at 1675788863677Writing region info on filesystem at 1675788863677Initializing all the Stores at 1675788863678 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788863678Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788863678Cleaning up temporary data from old regions at 1675788863685 (+7 ms)Cleaning up detritus from prior splits at 1675788863686 (+1 ms)Running coprocessor post-open hooks at 1675788863690 (+4 ms)Region opened successfully at 1675788863690 2023-02-07 16:54:23,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57., pid=43, masterSystemTime=1675788863668 2023-02-07 16:54:23,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,693 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,694 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788863693"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788863693"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788863693"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788863693"}]},"ts":"1675788863693"} 2023-02-07 16:54:23,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-07 16:54:23,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 in 178 msec 2023-02-07 16:54:23,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-07 16:54:23,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN in 336 msec 2023-02-07 16:54:23,699 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:23,699 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788863699"}]},"ts":"1675788863699"} 2023-02-07 16:54:23,701 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-07 16:54:23,718 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:23,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 442 msec 2023-02-07 16:54:23,780 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-07 16:54:23,782 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:48462, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-07 16:54:23,789 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:23,789 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:23,790 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:23,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 16:54:23,792 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 16:54:23,792 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:23,792 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 16:54:23,792 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:54:23,793 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,793 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:23,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:23,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:23,801 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:23,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:23,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:23,802 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,802 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:23,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:23,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:23,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:23,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:23,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:23,803 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:23,803 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,804 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. started... 2023-02-07 16:54:23,805 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:54:23,811 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,822 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,822 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:23,822 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,823 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,823 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:23,823 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/2214141826a440d98c4fb37aad55c802 2023-02-07 16:54:23,847 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/da5cc9e4ceae4dbeb565b04158d2969f 2023-02-07 16:54:23,852 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/2214141826a440d98c4fb37aad55c802 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 2023-02-07 16:54:23,857 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:23,858 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/da5cc9e4ceae4dbeb565b04158d2969f as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f 2023-02-07 16:54:23,862 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:23,863 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 58ms, sequenceid=6, compaction requested=false 2023-02-07 16:54:23,863 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-07 16:54:23,864 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:23,864 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:23,864 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:23,864 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:23,864 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,864 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:23,864 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:23,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,874 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:23,875 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,875 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 16:54:23,876 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7646bf35[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:23,876 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:23,885 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,885 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:23,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,885 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:23,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:23,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:23,886 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:23,896 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:23,896 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:23,896 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,896 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:23,896 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:23,896 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:23,896 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:23,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:23,906 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:23,906 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:23,906 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:23,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:23,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:23,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:23,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:23,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:23,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:54:23,908 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 16:54:23,908 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:23,908 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:23,908 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 16:54:23,908 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:23,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:23,917 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:23,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:23,918 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:23,918 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,920 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,920 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,920 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:23,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:23,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:23,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 16:54:23,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:23,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:23,928 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:23,927 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:23,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:23,928 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:23,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:23,928 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:23,929 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:23,929 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:23,929 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:23,929 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:23,929 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,029 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:24,029 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,029 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:24,049 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,049 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:24,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:24,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 16:54:24,051 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,052 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,052 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 16:54:24,052 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:54:24,052 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,052 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,064 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,064 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,064 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,064 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,065 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,066 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:24,067 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:24,067 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,067 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. started... 2023-02-07 16:54:24,067 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:24,074 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,079 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/0271733eba71488da02e97fb64a0ccb6 2023-02-07 16:54:24,085 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,085 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,085 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,085 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,085 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,092 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/1e728b13bfb449a2b819819480fdc429 2023-02-07 16:54:24,098 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/0271733eba71488da02e97fb64a0ccb6 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 2023-02-07 16:54:24,103 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:24,104 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/1e728b13bfb449a2b819819480fdc429 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 2023-02-07 16:54:24,109 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:24,110 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 43ms, sequenceid=19, compaction requested=false 2023-02-07 16:54:24,111 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:24,111 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:24,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:24,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,111 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,117 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,117 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,117 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,117 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 16:54:24,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d7fa371[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:24,119 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:54:24,119 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,128 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,128 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:24,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,128 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:24,171 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,171 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,171 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,171 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,171 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,171 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,190 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,190 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,190 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,190 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:24,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:54:24,194 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 16:54:24,194 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:24,194 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:24,194 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 16:54:24,194 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,202 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,202 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,203 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,203 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,204 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,212 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,212 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:24,212 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,212 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,213 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,213 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,213 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,213 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,213 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,213 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,313 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:24,313 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,313 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:24,329 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,329 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,330 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:24,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:24,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 16:54:24,332 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,332 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,332 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 16:54:24,332 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:54:24,332 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,333 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,338 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,338 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,338 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,338 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,338 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,338 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:24,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,340 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,340 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,340 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. started... 2023-02-07 16:54:24,341 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:24,348 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,355 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 2023-02-07 16:54:24,359 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,359 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,359 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,359 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,359 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,373 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/d795076ea8c64e9fbff38ffbdae9f1bc 2023-02-07 16:54:24,377 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 2023-02-07 16:54:24,381 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:24,382 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/d795076ea8c64e9fbff38ffbdae9f1bc as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc 2023-02-07 16:54:24,386 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:24,387 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 47ms, sequenceid=32, compaction requested=true 2023-02-07 16:54:24,387 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:24,387 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,387 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:24,388 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:24,388 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,388 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,388 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,401 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,401 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,402 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,402 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 16:54:24,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@720ffd30[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:24,404 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:54:24,404 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,412 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,412 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:24,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,412 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:24,422 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,422 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,422 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,432 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,432 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,432 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,432 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,443 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,443 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,443 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:24,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:54:24,446 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 16:54:24,446 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:24,446 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:24,446 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 16:54:24,446 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:24,453 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,453 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,453 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,453 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,453 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,454 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,454 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,455 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,456 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,456 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,456 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,488 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:24,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,488 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,589 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:24,589 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,589 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:24,607 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,608 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,608 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:24,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:24,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 16:54:24,610 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,610 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,610 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 16:54:24,610 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:54:24,611 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,611 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,632 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,633 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,633 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:24,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,633 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,634 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,635 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. started... 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:24,634 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,635 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,635 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,643 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,645 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/3c34d72b104e43ec85d91a91ead83b03 2023-02-07 16:54:24,653 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,653 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,654 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,654 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,654 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,656 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/b344732d53b84494adb92c6f4778b8d2 2023-02-07 16:54:24,661 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/3c34d72b104e43ec85d91a91ead83b03 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 2023-02-07 16:54:24,666 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:24,667 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/b344732d53b84494adb92c6f4778b8d2 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 2023-02-07 16:54:24,672 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:24,673 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 38ms, sequenceid=45, compaction requested=true 2023-02-07 16:54:24,673 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:24,673 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,673 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:24,673 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:24,673 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,673 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,673 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,685 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,685 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,686 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,686 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 16:54:24,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3796e377[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:24,688 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:54:24,688 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,696 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,696 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:24,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,696 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,697 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:24,697 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:24,706 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,706 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,706 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,717 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,717 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,717 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,717 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,717 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,717 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,717 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:24,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:54:24,720 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 16:54:24,720 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:24,721 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:24,721 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 16:54:24,721 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:24,727 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,727 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,727 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,727 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,727 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,727 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,728 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,728 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,728 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,729 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,730 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,730 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,731 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,738 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:24,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,738 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:24,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:24,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,739 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,812 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:54:24,839 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:24,839 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,839 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:24,860 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:24,861 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:24,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:24,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:24,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 16:54:24,864 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 16:54:24,864 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,864 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 16:54:24,864 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:54:24,864 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,865 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,874 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,875 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,875 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,875 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:24,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,875 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,875 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:24,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,876 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,876 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,876 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,877 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,877 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,878 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,879 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:24,879 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:24,879 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,879 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. started... 2023-02-07 16:54:24,880 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:24,885 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,893 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/ecb8ddab204148229d4b813cfc31414d 2023-02-07 16:54:24,895 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,895 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,896 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,896 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,896 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,904 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/10534b3bdc1a473fbb3b6b25494f0381 2023-02-07 16:54:24,909 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/ecb8ddab204148229d4b813cfc31414d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d 2023-02-07 16:54:24,914 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:24,915 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/10534b3bdc1a473fbb3b6b25494f0381 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 2023-02-07 16:54:24,919 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:24,920 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 41ms, sequenceid=58, compaction requested=true 2023-02-07 16:54:24,920 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:24,920 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:24,920 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:24,920 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:24,920 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,920 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:24,920 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:24,927 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,927 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,928 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 16:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@746a4da7[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:24,930 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:54:24,930 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:24,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:24,938 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:24,939 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:24,939 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:24,973 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,973 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,973 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,990 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,990 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,990 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,990 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,990 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:24,991 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:24,991 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:24,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:24,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:24,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:24,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:24,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:24,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:54:24,994 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 16:54:24,994 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:24,994 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:24,994 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 16:54:24,994 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,001 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,002 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,002 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:25,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,003 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:25,004 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,004 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,011 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,011 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,011 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,012 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:25,011 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,012 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 16:54:25,012 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,011 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,012 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,013 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,112 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:25,113 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:25,113 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:25,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:25,129 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:25,129 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:25,130 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:25,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:25,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-07 16:54:25,132 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-07 16:54:25,132 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:25,132 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-07 16:54:25,132 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:54:25,132 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,132 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,138 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,138 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,139 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:25,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:25,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:25,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-07 16:54:25,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:25,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:25,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-07 16:54:25,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:25,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:25,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:25,140 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:25,140 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:25,141 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. started... 2023-02-07 16:54:25,141 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:25,148 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,150 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/67957763eedd4640885419935b034a82 2023-02-07 16:54:25,159 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,159 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,159 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:25,159 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,159 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:25,162 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/ae5b0d43a86647559543ff11ce9bd955 2023-02-07 16:54:25,167 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/67957763eedd4640885419935b034a82 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 2023-02-07 16:54:25,171 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:25,172 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/ae5b0d43a86647559543ff11ce9bd955 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 2023-02-07 16:54:25,177 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:25,178 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 37ms, sequenceid=71, compaction requested=true 2023-02-07 16:54:25,178 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:25,178 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:25,178 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:25,178 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:25,178 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,178 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-07 16:54:25,178 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-07 16:54:25,190 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,190 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:25,191 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,191 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:25,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-07 16:54:25,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@146e0c70[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:25,193 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:54:25,193 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,201 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:25,201 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:25,201 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:25,201 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:25,201 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:25,201 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:54:25,202 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-07 16:54:25,202 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:25,202 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:25,202 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:25,211 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:25,212 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:25,212 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:25,211 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:25,211 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,211 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-07 16:54:25,212 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:25,212 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:25,212 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:25,212 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-07 16:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:54:25,215 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-07 16:54:25,215 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:25,215 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:25,215 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-07 16:54:25,215 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,222 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,222 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,223 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,223 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,224 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:25,225 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-07 16:54:25,225 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,259 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-07 16:54:25,259 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:25,259 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:25,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-07 16:54:25,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:25,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:25,259 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:25,260 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:25,260 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:25,260 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-07 16:54:25,260 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-07 16:54:25,360 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:25,360 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:25,360 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:25,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:25,622 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-07 16:54:25,623 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:25,625 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,627 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:25,627 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:25,627 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:25,875 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,875 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:25,875 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:25,875 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:25,875 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-07 16:54:25,875 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:25,876 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:25,876 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:25,876 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:25,876 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:25,878 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-07 16:54:25,878 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:25,878 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:25,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:25,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:25,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:25,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:25,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:25,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:25,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:25,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:25,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:25,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:25,882 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:25,882 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,882 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:25,882 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,882 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,882 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,883 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:25,883 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:25,884 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:25,884 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788895884 2023-02-07 16:54:25,886 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,886 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:25,889 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:25,889 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:25,889 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:25,889 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788925889 2023-02-07 16:54:25,889 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788985889 2023-02-07 16:54:25,890 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:25,890 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:25,892 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:25,893 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:25,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:25,893 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:26,140 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,140 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:26,140 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:26,140 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:26,140 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:26,140 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:26,141 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:26,141 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:26,141 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:26,142 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:26,143 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 16:54:26,143 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:26,143 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:26,144 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:26,144 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:26,144 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,144 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:26,144 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:26,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:26,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,146 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,146 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,146 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,146 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:26,147 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:26,148 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:26,148 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788896148 2023-02-07 16:54:26,148 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:26,149 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,152 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,152 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,152 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:26,152 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788926152 2023-02-07 16:54:26,152 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788986152 2023-02-07 16:54:26,152 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:54:26,152 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:26,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN}] 2023-02-07 16:54:26,155 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:26,155 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:26,155 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:26,403 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:26,403 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:26,404 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:26,405 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 16:54:26,406 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:26,406 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:26,406 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:26,406 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:26,407 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,407 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:26,407 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,407 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,408 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,408 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:26,408 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,409 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,409 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:26,409 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,410 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,410 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,410 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,410 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,410 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:26,411 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:26,411 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:26,412 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788896412 2023-02-07 16:54:26,413 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:26,413 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,415 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,415 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,416 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:26,416 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788926416 2023-02-07 16:54:26,416 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788986416 2023-02-07 16:54:26,416 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:26,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN}] 2023-02-07 16:54:26,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:26,419 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:26,420 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788866420"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788866420"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788866420"}]},"ts":"1675788866420"} 2023-02-07 16:54:26,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:26,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:26,422 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:26,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:26,666 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,666 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:26,666 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:26,667 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:26,667 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:26,667 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:26,667 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:26,667 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:26,667 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:26,668 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:54:26,670 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:26,670 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 16:54:26,670 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:26,670 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:26,671 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:26,671 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:26,671 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,671 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:26,672 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,672 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,672 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,673 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:26,673 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,673 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,673 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:26,674 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,674 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,674 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,674 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,675 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,675 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,675 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,675 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,675 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:26,676 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:26,676 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:26,676 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:26,677 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788896677 2023-02-07 16:54:26,678 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,678 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:26,680 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,681 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,681 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:26,681 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788926681 2023-02-07 16:54:26,681 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788986681 2023-02-07 16:54:26,681 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:26,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:26,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788866681"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788866681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788866681"}]},"ts":"1675788866681"} 2023-02-07 16:54:26,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:26,683 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:26,837 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:26,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing ab96ff60256f44e85e21a1807242ec57, disabling compactions & flushes 2023-02-07 16:54:26,837 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:26,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:26,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. after waiting 0 ms 2023-02-07 16:54:26,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:26,837 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing ab96ff60256f44e85e21a1807242ec57 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:54:26,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/f764f47c0e324ce3b53e19346852f051 2023-02-07 16:54:26,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/1609df4040d94b59a8523d35af76ab7e 2023-02-07 16:54:26,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/f764f47c0e324ce3b53e19346852f051 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 2023-02-07 16:54:26,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:54:26,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/1609df4040d94b59a8523d35af76ab7e as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e 2023-02-07 16:54:26,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:54:26,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 40ms, sequenceid=82, compaction requested=true 2023-02-07 16:54:26,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:54:26,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:26,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:26,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for ab96ff60256f44e85e21a1807242ec57: Waiting for close lock at 1675788866837Running coprocessor pre-close hooks at 1675788866837Disabling compacts and flushes for region at 1675788866837Disabling writes for close at 1675788866837Obtaining lock to block concurrent updates at 1675788866837Preparing flush snapshotting stores in ab96ff60256f44e85e21a1807242ec57 at 1675788866837Finished memstore snapshotting testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675788866838 (+1 ms)Flushing stores of testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. at 1675788866838Flushing ab96ff60256f44e85e21a1807242ec57/cf1: creating writer at 1675788866839 (+1 ms)Flushing ab96ff60256f44e85e21a1807242ec57/cf1: appending metadata at 1675788866841 (+2 ms)Flushing ab96ff60256f44e85e21a1807242ec57/cf1: closing flushed file at 1675788866841Flushing ab96ff60256f44e85e21a1807242ec57/cf2: creating writer at 1675788866852 (+11 ms)Flushing ab96ff60256f44e85e21a1807242ec57/cf2: appending metadata at 1675788866854 (+2 ms)Flushing ab96ff60256f44e85e21a1807242ec57/cf2: closing flushed file at 1675788866854Flushing ab96ff60256f44e85e21a1807242ec57/cf1: reopening flushed file at 1675788866865 (+11 ms)Flushing ab96ff60256f44e85e21a1807242ec57/cf2: reopening flushed file at 1675788866871 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ab96ff60256f44e85e21a1807242ec57 in 40ms, sequenceid=82, compaction requested=true at 1675788866877 (+6 ms)Writing region close event to WAL at 1675788866878 (+1 ms)Running coprocessor post-close hooks at 1675788866881 (+3 ms)Closed at 1675788866881 2023-02-07 16:54:26,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:26,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED 2023-02-07 16:54:26,885 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788866884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788866884"}]},"ts":"1675788866884"} 2023-02-07 16:54:26,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:26,886 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:26,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:26,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:26,931 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,931 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:26,931 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:26,932 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:26,932 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:26,932 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:26,932 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:26,932 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:26,932 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:26,933 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:26,934 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:26,935 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,935 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:26,935 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,935 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:26,936 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,936 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:26,936 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,936 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:26,936 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:26,937 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,937 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,937 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:26,937 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,937 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,937 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,937 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:26,937 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,937 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,937 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:26,937 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:26,938 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:26,939 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:26,939 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:26,939 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788896939 2023-02-07 16:54:26,940 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:26,940 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:26,942 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,942 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:26,942 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:26,942 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788926942 2023-02-07 16:54:26,942 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788986942 2023-02-07 16:54:26,943 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:26,943 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:26,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788866943"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788866943"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788866943"}]},"ts":"1675788866943"} 2023-02-07 16:54:26,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:26,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-07 16:54:26,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 in 261 msec 2023-02-07 16:54:26,947 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:26,947 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:26,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:26,947 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:27,193 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:27,193 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:27,194 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:27,194 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:27,195 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:27,195 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-07 16:54:27,195 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:27,196 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:27,196 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:27,196 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:27,196 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:27,196 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,197 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:27,197 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,197 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,197 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,197 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:27,197 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,198 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,198 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:27,199 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,199 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:27,199 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:27,199 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:27,199 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,199 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,199 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,199 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,200 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,200 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:27,200 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:27,201 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:27,201 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:27,201 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788897201 2023-02-07 16:54:27,202 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:27,202 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:27,204 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:27,204 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:27,204 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:27,204 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788927204 2023-02-07 16:54:27,204 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788987204 2023-02-07 16:54:27,205 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:27,205 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:27,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:27,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-07 16:54:27,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN in 791 msec 2023-02-07 16:54:27,213 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:27,213 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:27,213 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:27,455 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:27,455 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:27,456 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:27,456 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:27,457 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:27,457 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:27,457 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:27,457 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:27,458 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:27,458 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:27,458 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:27,458 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,458 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:27,459 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,459 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,459 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,459 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:27,459 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,459 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,460 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,460 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:27,461 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:27,462 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:27,464 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788897464 2023-02-07 16:54:27,465 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,465 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:27,467 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:27,468 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:27,468 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:27,468 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788927468 2023-02-07 16:54:27,468 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788987468 2023-02-07 16:54:27,468 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:27,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,473 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:27,483 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ab96ff60256f44e85e21a1807242ec57, threads=14 2023-02-07 16:54:27,484 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,484 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,484 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,484 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,484 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,485 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,485 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,485 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,485 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,485 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,489 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,489 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,489 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,489 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,693 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,695 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,695 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,697 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,705 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,705 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,718 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,720 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,721 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,722 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,724 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,733 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,740 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,743 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,743 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ab96ff60256f44e85e21a1807242ec57 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:54:27,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:27,789 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:27,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:27,968 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,968 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:27,968 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:27,968 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:27,969 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:27,969 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:27,969 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:27,969 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:27,969 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:27,970 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:27,970 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 16:54:27,971 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:27,971 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:27,971 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:27,971 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:27,971 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:27,971 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:27,972 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,972 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:27,972 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,972 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:27,972 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,972 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:27,973 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,973 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:27,973 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:27,973 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,973 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:27,974 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:27,974 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:27,974 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,974 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,974 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,974 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,974 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:27,974 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:27,975 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:27,976 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:27,977 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788897977 2023-02-07 16:54:27,979 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:27,979 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:27,982 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:27,982 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:27,982 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:27,982 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788927982 2023-02-07 16:54:27,982 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788987982 2023-02-07 16:54:27,982 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:27,992 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ab96ff60256f44e85e21a1807242ec57, threads=14 2023-02-07 16:54:27,993 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,994 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,994 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,996 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,996 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,997 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,997 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,997 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,997 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:27,997 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,005 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,005 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,006 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,006 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,181 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,181 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,194 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,206 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,216 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,217 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,222 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,225 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,237 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,257 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,257 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,257 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,258 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,261 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 for region: ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:28,261 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ab96ff60256f44e85e21a1807242ec57 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:54:28,294 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:28,299 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/495e15228309be9b15c671620a05b627/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:54:28,301 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/c77b9139a0c759abb3f7951754110a14/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:54:28,301 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:28,301 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:28,301 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:28,483 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,483 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:28,483 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:28,483 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:28,483 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:28,484 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:28,484 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:28,484 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:28,484 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:28,485 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:28,485 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 16:54:28,486 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:28,486 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:28,486 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:28,486 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:28,486 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:28,486 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:28,487 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:28,487 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:28,487 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:28,487 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:28,488 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:28,488 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:28,488 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:28,488 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:28,488 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:28,489 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,489 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:28,489 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:28,491 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:28,491 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,491 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,491 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,491 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:28,491 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:28,492 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:28,492 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:28,493 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:28,493 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788898493 2023-02-07 16:54:28,494 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:28,494 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,497 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:28,497 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:28,497 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:28,497 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788928497 2023-02-07 16:54:28,497 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788988497 2023-02-07 16:54:28,497 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:28,500 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:28,504 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:28,504 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:28,504 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:28,748 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,748 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:28,748 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:28,748 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:28,750 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:28,751 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:28,751 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:28,751 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:28,751 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:28,752 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:28,752 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 16:54:28,752 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:28,752 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:28,753 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:28,753 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:28,753 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:28,753 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:28,754 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:28,754 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:28,754 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:28,754 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:28,754 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:28,755 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:28,755 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:28,755 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:28,755 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:28,756 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,756 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:28,756 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:28,756 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:28,756 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,756 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,757 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,757 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:28,757 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:28,757 DEBUG [Listener at localhost.localdomain/34435] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,757 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:28,758 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:28,758 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:28,761 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788898761 2023-02-07 16:54:28,763 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:28,763 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:28,766 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:28,766 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:28,766 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:28,766 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788928766 2023-02-07 16:54:28,766 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788988766 2023-02-07 16:54:28,766 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:28,767 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:28,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:28,777 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:28,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:28,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:29,018 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,018 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:29,018 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:29,018 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:29,018 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:29,018 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:29,018 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:29,019 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:29,019 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:29,019 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:29,020 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-07 16:54:29,020 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:29,020 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:29,020 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:29,021 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:29,021 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:29,021 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:29,022 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,022 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:29,022 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,022 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,023 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,023 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:29,023 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,023 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,024 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:29,025 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,025 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:29,025 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:29,025 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:29,026 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,026 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,026 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,026 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,026 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,026 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:29,027 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:29,028 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:29,028 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788899028 2023-02-07 16:54:29,031 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:29,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,033 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,033 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,034 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:29,034 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788929034 2023-02-07 16:54:29,034 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788989034 2023-02-07 16:54:29,034 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:29,034 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:29,037 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:29,037 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:29,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:29,037 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:29,285 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,285 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:29,285 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:29,285 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:29,285 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:29,286 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:29,286 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:29,286 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:29,286 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:29,286 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:29,287 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 16:54:29,287 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:29,287 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:29,287 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:29,288 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:29,288 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:29,288 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:29,288 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,288 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:29,288 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,289 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,289 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,289 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:29,289 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,289 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,289 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:29,290 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,290 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:29,290 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:29,290 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:29,290 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,290 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,290 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,290 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,290 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,291 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-07 16:54:29,291 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:29,292 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:29,292 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788899292 2023-02-07 16:54:29,294 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:29,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,296 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,296 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,296 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:29,296 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788929296 2023-02-07 16:54:29,296 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788989296 2023-02-07 16:54:29,297 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:29,297 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:29,300 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:29,300 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:29,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:29,300 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:29,547 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:29,547 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:29,548 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:29,548 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:29,549 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 16:54:29,549 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:29,549 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:29,549 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:29,550 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:29,550 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:29,550 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:29,550 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,550 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:29,550 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,551 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,551 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,551 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:29,551 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,551 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,551 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:29,552 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,552 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:29,552 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:29,552 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:29,552 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,552 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,552 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,552 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,552 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,552 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 16:54:29,553 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:29,554 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:29,554 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788899554 2023-02-07 16:54:29,556 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:29,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,558 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,559 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,559 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:29,559 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788929559 2023-02-07 16:54:29,559 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788989559 2023-02-07 16:54:29,559 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:54:29,559 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:29,564 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN] 2023-02-07 16:54:29,564 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:29,564 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:29,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:29,564 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:29,809 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,809 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:29,810 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:29,810 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:29,810 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:29,810 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:29,810 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:29,810 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:29,810 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:29,811 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:29,812 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 16:54:29,812 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:54:29,812 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:54:29,812 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 16:54:29,813 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:29,813 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:29,813 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:29,814 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:29,814 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,814 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:29,814 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,814 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:29,815 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,815 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:29,815 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,815 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:29,816 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:29,817 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,817 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:29,817 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:29,817 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:29,817 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:29,817 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,817 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,818 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,818 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,818 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,818 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:29,818 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:54:29,819 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:29,819 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:29,820 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:29,820 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788899820 2023-02-07 16:54:29,822 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:29,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:29,823 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:29,825 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,826 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:29,826 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:29,826 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788929826 2023-02-07 16:54:29,826 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788989826 2023-02-07 16:54:29,826 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:54:29,826 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:29,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:29,831 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:29,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:29,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:30,076 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:30,077 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:30,077 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:30,078 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:30,079 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:54:30,079 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 16:54:30,080 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:30,080 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:30,080 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:30,080 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:30,080 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,081 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:30,081 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,081 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,082 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,082 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:30,082 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,082 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,082 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:30,083 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,083 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,083 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:30,083 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,083 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,084 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,084 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:30,084 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:30,085 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:30,085 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:30,085 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:30,086 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788900086 2023-02-07 16:54:30,087 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:30,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,088 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,090 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,091 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,091 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:30,091 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788930091 2023-02-07 16:54:30,091 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788990091 2023-02-07 16:54:30,091 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:54:30,091 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:30,094 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:30,094 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:30,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:30,094 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:30,341 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,341 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:30,342 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:30,342 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:30,343 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:30,343 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:30,343 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:30,343 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:30,343 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:30,344 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:30,345 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 16:54:30,345 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 16:54:30,346 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:30,346 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:30,346 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:30,346 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:30,347 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,347 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:30,347 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,347 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,348 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,348 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:30,348 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,348 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,349 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:30,349 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,349 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,349 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:30,350 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,350 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,350 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,350 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:30,350 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:30,351 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:30,351 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:30,351 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:30,352 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788900352 2023-02-07 16:54:30,354 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:30,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,355 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,357 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,358 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,358 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:30,358 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788930358 2023-02-07 16:54:30,358 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788990358 2023-02-07 16:54:30,358 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:54:30,358 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:30,362 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:30,362 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-07 16:54:30,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:30,362 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:30,608 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:30,609 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:30,609 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:30,610 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:30,611 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 16:54:30,612 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-07 16:54:30,612 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:30,612 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:30,613 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:30,613 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:30,613 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,613 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:30,613 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,614 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,614 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,614 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:30,614 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,615 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,615 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:30,615 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,615 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,616 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:30,616 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,616 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,616 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,616 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:54:30,616 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-07 16:54:30,617 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:30,617 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:30,618 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:30,618 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788900618 2023-02-07 16:54:30,619 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:30,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,623 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,624 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,625 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,625 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:30,625 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788930625 2023-02-07 16:54:30,625 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788990625 2023-02-07 16:54:30,625 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-07 16:54:30,625 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:54:30,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 exec-time=5.0010 sec 2023-02-07 16:54:30,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,630 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=true 2023-02-07 16:54:30,780 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:30,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:30,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788870781"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788870781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788870781"}]},"ts":"1675788870781"} 2023-02-07 16:54:30,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:30,784 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:54:30,784 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:54:30,784 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:54:30,875 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:30,876 WARN [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-07 16:54:30,876 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:54:30,876 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:54:30,876 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:54:30,877 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:54:30,877 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:54:30,877 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:54:30,877 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:54:30,877 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:54:30,878 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:54:30,879 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-07 16:54:30,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:54:30,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:54:30,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:54:30,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:54:30,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:54:30,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:30,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:54:30,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:54:30,881 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,881 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-07 16:54:30,882 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:54:30,883 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:54:30,883 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57 to restore RIT 2023-02-07 16:54:30,883 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788900883 2023-02-07 16:54:30,884 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:54:30,884 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:54:30,886 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,886 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=ab96ff60256f44e85e21a1807242ec57, regionState=OPENING, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:54:30,886 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:54:30,886 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675788930886 2023-02-07 16:54:30,886 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675788990886 2023-02-07 16:54:30,887 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:54:30,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:30,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788870887"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788870887"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788870887"}]},"ts":"1675788870887"} 2023-02-07 16:54:30,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:31,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ab96ff60256f44e85e21a1807242ec57, NAME => 'testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:31,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:31,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:31,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,046 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,047 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1 2023-02-07 16:54:31,047 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1 2023-02-07 16:54:31,047 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab96ff60256f44e85e21a1807242ec57 columnFamilyName cf1 2023-02-07 16:54:31,053 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 2023-02-07 16:54:31,057 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 2023-02-07 16:54:31,060 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 2023-02-07 16:54:31,063 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 2023-02-07 16:54:31,066 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 2023-02-07 16:54:31,070 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d 2023-02-07 16:54:31,074 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 2023-02-07 16:54:31,074 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(310): Store=ab96ff60256f44e85e21a1807242ec57/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:31,075 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,076 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2 2023-02-07 16:54:31,076 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2 2023-02-07 16:54:31,076 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab96ff60256f44e85e21a1807242ec57 columnFamilyName cf2 2023-02-07 16:54:31,083 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 2023-02-07 16:54:31,087 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e 2023-02-07 16:54:31,091 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 2023-02-07 16:54:31,095 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 2023-02-07 16:54:31,099 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 2023-02-07 16:54:31,104 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc 2023-02-07 16:54:31,108 DEBUG [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f 2023-02-07 16:54:31,108 INFO [StoreOpener-ab96ff60256f44e85e21a1807242ec57-1] regionserver.HStore(310): Store=ab96ff60256f44e85e21a1807242ec57/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:31,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.splits 2023-02-07 16:54:31,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:31,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened ab96ff60256f44e85e21a1807242ec57; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69113161, jitterRate=0.029866352677345276}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:31,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for ab96ff60256f44e85e21a1807242ec57: Running coprocessor pre-open hook at 1675788871045Writing region info on filesystem at 1675788871045Initializing all the Stores at 1675788871046 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788871046Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788871046Cleaning up temporary data from old regions at 1675788871110 (+64 ms)Cleaning up detritus from prior splits at 1675788871111 (+1 ms)Running coprocessor post-open hooks at 1675788871115 (+4 ms)Region opened successfully at 1675788871115 2023-02-07 16:54:31,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57., pid=48, masterSystemTime=1675788871041 2023-02-07 16:54:31,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:31,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:31,117 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:31,117 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:31,117 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 16:54:31,117 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 16:54:31,118 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1912): ab96ff60256f44e85e21a1807242ec57/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:31,118 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1912): ab96ff60256f44e85e21a1807242ec57/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:31,118 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ab96ff60256f44e85e21a1807242ec57/cf2 in testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,118 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ab96ff60256f44e85e21a1807242ec57/cf1 in testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,118 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp, totalSize=34.9 K 2023-02-07 16:54:31,118 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp, totalSize=34.9 K 2023-02-07 16:54:31,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,119 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 2214141826a440d98c4fb37aad55c802, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788863783 2023-02-07 16:54:31,119 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting da5cc9e4ceae4dbeb565b04158d2969f, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788863783 2023-02-07 16:54:31,120 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 0271733eba71488da02e97fb64a0ccb6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788864031 2023-02-07 16:54:31,120 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 1e728b13bfb449a2b819819480fdc429, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788864031 2023-02-07 16:54:31,120 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:31,120 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788871120"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788871120"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788871120"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788871120"}]},"ts":"1675788871120"} 2023-02-07 16:54:31,120 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 0bb3819f3e8d43c8a0a1a90113c582f6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788864315 2023-02-07 16:54:31,120 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting d795076ea8c64e9fbff38ffbdae9f1bc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788864315 2023-02-07 16:54:31,120 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 3c34d72b104e43ec85d91a91ead83b03, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788864591 2023-02-07 16:54:31,120 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting b344732d53b84494adb92c6f4778b8d2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788864591 2023-02-07 16:54:31,121 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting ecb8ddab204148229d4b813cfc31414d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788864841 2023-02-07 16:54:31,121 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 10534b3bdc1a473fbb3b6b25494f0381, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788864841 2023-02-07 16:54:31,121 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 67957763eedd4640885419935b034a82, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788865114 2023-02-07 16:54:31,121 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting ae5b0d43a86647559543ff11ce9bd955, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788865114 2023-02-07 16:54:31,121 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting f764f47c0e324ce3b53e19346852f051, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788865362 2023-02-07 16:54:31,121 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 1609df4040d94b59a8523d35af76ab7e, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788865362 2023-02-07 16:54:31,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-07 16:54:31,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 in 233 msec 2023-02-07 16:54:31,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN in 1.5630 sec 2023-02-07 16:54:31,142 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] throttle.PressureAwareThroughputController(145): ab96ff60256f44e85e21a1807242ec57#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:31,147 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] throttle.PressureAwareThroughputController(145): ab96ff60256f44e85e21a1807242ec57#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:31,165 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf1/31b78d8f502d4a94a869bfac5545c527 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/31b78d8f502d4a94a869bfac5545c527 2023-02-07 16:54:31,165 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/.tmp/cf2/c97e46107e20482895aac31baef6f23a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/c97e46107e20482895aac31baef6f23a 2023-02-07 16:54:31,170 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ab96ff60256f44e85e21a1807242ec57/cf1 of ab96ff60256f44e85e21a1807242ec57 into 31b78d8f502d4a94a869bfac5545c527(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:31,170 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ab96ff60256f44e85e21a1807242ec57/cf2 of ab96ff60256f44e85e21a1807242ec57 into c97e46107e20482895aac31baef6f23a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:31,170 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:31,171 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ab96ff60256f44e85e21a1807242ec57: 2023-02-07 16:54:31,171 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57., storeName=ab96ff60256f44e85e21a1807242ec57/cf1, priority=9, startTime=1675788871116; duration=0sec 2023-02-07 16:54:31,171 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57., storeName=ab96ff60256f44e85e21a1807242ec57/cf2, priority=9, startTime=1675788871116; duration=0sec 2023-02-07 16:54:31,172 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:31,172 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:31,238 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-07 16:54:31,241 DEBUG [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-07 16:54:31,242 INFO [Listener at localhost.localdomain/34435] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-07 16:54:31,245 INFO [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-07 16:54:31,245 INFO [Listener at localhost.localdomain/34435] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-07 16:54:31,245 INFO [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-07 16:54:31,261 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:31,261 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,261 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,263 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,263 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,263 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-07 16:54:31,264 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRollbackAndDoubleExecution 2023-02-07 16:54:31,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,267 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,267 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-07 16:54:31,268 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788871268"}]},"ts":"1675788871268"} 2023-02-07 16:54:31,269 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-07 16:54:31,316 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-07 16:54:31,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN}] 2023-02-07 16:54:31,318 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN 2023-02-07 16:54:31,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:31,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788871319"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788871319"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788871319"}]},"ts":"1675788871319"} 2023-02-07 16:54:31,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:31,368 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,368 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-07 16:54:31,472 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing ab96ff60256f44e85e21a1807242ec57, disabling compactions & flushes 2023-02-07 16:54:31,472 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. after waiting 0 ms 2023-02-07 16:54:31,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,473 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051] to archive 2023-02-07 16:54:31,474 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:31,475 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/2214141826a440d98c4fb37aad55c802 2023-02-07 16:54:31,477 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0271733eba71488da02e97fb64a0ccb6 2023-02-07 16:54:31,478 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/0bb3819f3e8d43c8a0a1a90113c582f6 2023-02-07 16:54:31,479 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/3c34d72b104e43ec85d91a91ead83b03 2023-02-07 16:54:31,480 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/ecb8ddab204148229d4b813cfc31414d 2023-02-07 16:54:31,481 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/67957763eedd4640885419935b034a82 2023-02-07 16:54:31,483 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/f764f47c0e324ce3b53e19346852f051 2023-02-07 16:54:31,486 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e] to archive 2023-02-07 16:54:31,487 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:31,489 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/da5cc9e4ceae4dbeb565b04158d2969f 2023-02-07 16:54:31,490 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1e728b13bfb449a2b819819480fdc429 2023-02-07 16:54:31,491 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/d795076ea8c64e9fbff38ffbdae9f1bc 2023-02-07 16:54:31,492 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/b344732d53b84494adb92c6f4778b8d2 2023-02-07 16:54:31,493 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/10534b3bdc1a473fbb3b6b25494f0381 2023-02-07 16:54:31,495 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/ae5b0d43a86647559543ff11ce9bd955 2023-02-07 16:54:31,495 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/1609df4040d94b59a8523d35af76ab7e 2023-02-07 16:54:31,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:54:31,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:31,500 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57. 2023-02-07 16:54:31,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for ab96ff60256f44e85e21a1807242ec57: Waiting for close lock at 1675788871472Running coprocessor pre-close hooks at 1675788871472Disabling compacts and flushes for region at 1675788871472Disabling writes for close at 1675788871472Writing region close event to WAL at 1675788871497 (+25 ms)Running coprocessor post-close hooks at 1675788871500 (+3 ms)Closed at 1675788871500 2023-02-07 16:54:31,502 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,502 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ab96ff60256f44e85e21a1807242ec57, regionState=CLOSED 2023-02-07 16:54:31,503 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788871502"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788871502"}]},"ts":"1675788871502"} 2023-02-07 16:54:31,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-07 16:54:31,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure ab96ff60256f44e85e21a1807242ec57, server=jenkins-hbase12.apache.org,43011,1675788846300 in 184 msec 2023-02-07 16:54:31,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-07 16:54:31,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, UNASSIGN in 189 msec 2023-02-07 16:54:31,509 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788871509"}]},"ts":"1675788871509"} 2023-02-07 16:54:31,510 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-07 16:54:31,516 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-07 16:54:31,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 253 msec 2023-02-07 16:54:31,570 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,570 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-07 16:54:31,572 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-07 16:54:31,572 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,572 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,573 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRollbackAndDoubleExecution 2023-02-07 16:54:31,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,575 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,576 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,576 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-07 16:54:31,580 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,582 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/recovered.edits] 2023-02-07 16:54:31,585 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/31b78d8f502d4a94a869bfac5545c527 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf1/31b78d8f502d4a94a869bfac5545c527 2023-02-07 16:54:31,587 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/c97e46107e20482895aac31baef6f23a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/cf2/c97e46107e20482895aac31baef6f23a 2023-02-07 16:54:31,590 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57/recovered.edits/90.seqid 2023-02-07 16:54:31,591 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackAndDoubleExecution/ab96ff60256f44e85e21a1807242ec57 2023-02-07 16:54:31,591 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-07 16:54:31,594 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,596 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-07 16:54:31,597 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-07 16:54:31,599 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,599 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-07 16:54:31,599 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788871599"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:31,600 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-07 16:54:31,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ab96ff60256f44e85e21a1807242ec57, NAME => 'testRollbackAndDoubleExecution,,1675788863276.ab96ff60256f44e85e21a1807242ec57.', STARTKEY => '', ENDKEY => ''}] 2023-02-07 16:54:31,600 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-07 16:54:31,601 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788871600"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:31,602 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-07 16:54:31,612 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:54:31,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 39 msec 2023-02-07 16:54:31,677 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,677 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-07 16:54:31,678 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-07 16:54:31,691 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=404 (was 403) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46834 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60758 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:47928 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:35136 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:38154 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60752 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:37116 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:47912 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:37516 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60210 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:36962 [Waiting for operation #44] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46866 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase12:43011-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60604 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:38158 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:36966 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46744 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 859) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=351 (was 388), ProcessCount=174 (was 174), AvailableMemoryMB=3054 (was 3073) 2023-02-07 16:54:31,702 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=404, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=174, AvailableMemoryMB=3054 2023-02-07 16:54:31,703 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:31,703 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:31,703 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:31,716 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:54:31,717 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:54:31,718 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:31,720 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:31,721 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:31,722 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:31,723 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d empty. 2023-02-07 16:54:31,723 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:31,723 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-07 16:54:31,737 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:31,738 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 28b4cc61e5f5c6a50e245a62a87d6c2d, NAME => 'testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:31,747 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:31,747 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 28b4cc61e5f5c6a50e245a62a87d6c2d, disabling compactions & flushes 2023-02-07 16:54:31,747 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:31,747 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:31,747 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. after waiting 0 ms 2023-02-07 16:54:31,747 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:31,747 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:31,747 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 28b4cc61e5f5c6a50e245a62a87d6c2d: Waiting for close lock at 1675788871747Disabling compacts and flushes for region at 1675788871747Disabling writes for close at 1675788871747Writing region close event to WAL at 1675788871747Closed at 1675788871747 2023-02-07 16:54:31,750 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:31,751 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675788871750"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788871750"}]},"ts":"1675788871750"} 2023-02-07 16:54:31,753 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:31,754 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:31,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788871754"}]},"ts":"1675788871754"} 2023-02-07 16:54:31,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-07 16:54:31,769 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:31,770 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:31,770 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:31,771 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:31,771 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:31,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, ASSIGN}] 2023-02-07 16:54:31,773 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, ASSIGN 2023-02-07 16:54:31,773 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:31,923 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:31,925 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=28b4cc61e5f5c6a50e245a62a87d6c2d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:31,925 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675788871924"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788871924"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788871924"}]},"ts":"1675788871924"} 2023-02-07 16:54:31,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 28b4cc61e5f5c6a50e245a62a87d6c2d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:32,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 28b4cc61e5f5c6a50e245a62a87d6c2d, NAME => 'testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:32,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,083 INFO [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,084 DEBUG [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/cf1 2023-02-07 16:54:32,085 DEBUG [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/cf1 2023-02-07 16:54:32,085 INFO [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 28b4cc61e5f5c6a50e245a62a87d6c2d columnFamilyName cf1 2023-02-07 16:54:32,086 INFO [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] regionserver.HStore(310): Store=28b4cc61e5f5c6a50e245a62a87d6c2d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:32,086 INFO [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,087 DEBUG [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/cf2 2023-02-07 16:54:32,087 DEBUG [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/cf2 2023-02-07 16:54:32,088 INFO [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 28b4cc61e5f5c6a50e245a62a87d6c2d columnFamilyName cf2 2023-02-07 16:54:32,089 INFO [StoreOpener-28b4cc61e5f5c6a50e245a62a87d6c2d-1] regionserver.HStore(310): Store=28b4cc61e5f5c6a50e245a62a87d6c2d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:32,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:32,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:32,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 28b4cc61e5f5c6a50e245a62a87d6c2d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71819007, jitterRate=0.07018660008907318}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:32,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 28b4cc61e5f5c6a50e245a62a87d6c2d: Running coprocessor pre-open hook at 1675788872082Writing region info on filesystem at 1675788872082Initializing all the Stores at 1675788872083 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788872083Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788872083Cleaning up temporary data from old regions at 1675788872091 (+8 ms)Cleaning up detritus from prior splits at 1675788872092 (+1 ms)Running coprocessor post-open hooks at 1675788872097 (+5 ms)Region opened successfully at 1675788872097 2023-02-07 16:54:32,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d., pid=55, masterSystemTime=1675788872078 2023-02-07 16:54:32,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,100 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=28b4cc61e5f5c6a50e245a62a87d6c2d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,100 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675788872100"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788872100"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788872100"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788872100"}]},"ts":"1675788872100"} 2023-02-07 16:54:32,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-07 16:54:32,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 28b4cc61e5f5c6a50e245a62a87d6c2d, server=jenkins-hbase12.apache.org,41937,1675788846347 in 176 msec 2023-02-07 16:54:32,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-07 16:54:32,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, ASSIGN in 332 msec 2023-02-07 16:54:32,106 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:32,106 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788872106"}]},"ts":"1675788872106"} 2023-02-07 16:54:32,107 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-07 16:54:32,212 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:32,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 495 msec 2023-02-07 16:54:32,219 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,220 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, UNASSIGN}] 2023-02-07 16:54:32,224 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, UNASSIGN 2023-02-07 16:54:32,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=28b4cc61e5f5c6a50e245a62a87d6c2d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675788872225"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788872225"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788872225"}]},"ts":"1675788872225"} 2023-02-07 16:54:32,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 28b4cc61e5f5c6a50e245a62a87d6c2d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:32,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,380 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 28b4cc61e5f5c6a50e245a62a87d6c2d, disabling compactions & flushes 2023-02-07 16:54:32,380 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,380 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,380 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. after waiting 0 ms 2023-02-07 16:54:32,380 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,383 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:54:32,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-07 16:54:32,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:32,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d. 2023-02-07 16:54:32,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 28b4cc61e5f5c6a50e245a62a87d6c2d: Waiting for close lock at 1675788872379Running coprocessor pre-close hooks at 1675788872379Disabling compacts and flushes for region at 1675788872379Disabling writes for close at 1675788872380 (+1 ms)Writing region close event to WAL at 1675788872381 (+1 ms)Running coprocessor post-close hooks at 1675788872384 (+3 ms)Closed at 1675788872384 2023-02-07 16:54:32,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:32,386 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=28b4cc61e5f5c6a50e245a62a87d6c2d, regionState=CLOSED 2023-02-07 16:54:32,386 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675788872386"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788872386"}]},"ts":"1675788872386"} 2023-02-07 16:54:32,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-07 16:54:32,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 28b4cc61e5f5c6a50e245a62a87d6c2d, server=jenkins-hbase12.apache.org,41937,1675788846347 in 161 msec 2023-02-07 16:54:32,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-07 16:54:32,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=28b4cc61e5f5c6a50e245a62a87d6c2d, UNASSIGN in 165 msec 2023-02-07 16:54:32,425 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-07 16:54:32,427 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-07 16:54:32,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675788872429"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675788872429"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675788872429"}]},"ts":"1675788872429"} 2023-02-07 16:54:32,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788872429"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788872429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788872429"}]},"ts":"1675788872429"} 2023-02-07 16:54:32,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788872429"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788872429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788872429"}]},"ts":"1675788872429"} 2023-02-07 16:54:32,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, ASSIGN}] 2023-02-07 16:54:32,434 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, ASSIGN 2023-02-07 16:54:32,435 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:32,435 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, ASSIGN 2023-02-07 16:54:32,436 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:32,585 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:54:32,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=31438e1bba921e7d9b8bb86997119dbb, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,587 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788872587"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788872587"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788872587"}]},"ts":"1675788872587"} 2023-02-07 16:54:32,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 31438e1bba921e7d9b8bb86997119dbb, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:32,592 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=1a6e4d9b038ccb4c495f43570894b413, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,592 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788872592"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788872592"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788872592"}]},"ts":"1675788872592"} 2023-02-07 16:54:32,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 1a6e4d9b038ccb4c495f43570894b413, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:32,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:32,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1a6e4d9b038ccb4c495f43570894b413, NAME => 'testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 16:54:32,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:32,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:32,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,746 INFO [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,748 DEBUG [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/cf1 2023-02-07 16:54:32,748 DEBUG [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/cf1 2023-02-07 16:54:32,748 INFO [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1a6e4d9b038ccb4c495f43570894b413 columnFamilyName cf1 2023-02-07 16:54:32,748 INFO [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] regionserver.HStore(310): Store=1a6e4d9b038ccb4c495f43570894b413/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:32,749 INFO [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,750 DEBUG [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/cf2 2023-02-07 16:54:32,750 DEBUG [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/cf2 2023-02-07 16:54:32,750 INFO [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1a6e4d9b038ccb4c495f43570894b413 columnFamilyName cf2 2023-02-07 16:54:32,751 INFO [StoreOpener-1a6e4d9b038ccb4c495f43570894b413-1] regionserver.HStore(310): Store=1a6e4d9b038ccb4c495f43570894b413/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:32,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:32,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:32,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1a6e4d9b038ccb4c495f43570894b413; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64791387, jitterRate=-0.034533098340034485}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:32,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1a6e4d9b038ccb4c495f43570894b413: Running coprocessor pre-open hook at 1675788872745Writing region info on filesystem at 1675788872745Initializing all the Stores at 1675788872746 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788872746Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788872746Cleaning up temporary data from old regions at 1675788872753 (+7 ms)Cleaning up detritus from prior splits at 1675788872754 (+1 ms)Running coprocessor post-open hooks at 1675788872756 (+2 ms)Region opened successfully at 1675788872756 2023-02-07 16:54:32,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413., pid=62, masterSystemTime=1675788872741 2023-02-07 16:54:32,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:32,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:32,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:32,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 31438e1bba921e7d9b8bb86997119dbb, NAME => 'testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 16:54:32,759 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=1a6e4d9b038ccb4c495f43570894b413, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:32,759 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788872759"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788872759"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788872759"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788872759"}]},"ts":"1675788872759"} 2023-02-07 16:54:32,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:32,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,761 INFO [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-07 16:54:32,762 DEBUG [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/cf1 2023-02-07 16:54:32,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 1a6e4d9b038ccb4c495f43570894b413, server=jenkins-hbase12.apache.org,41937,1675788846347 in 167 msec 2023-02-07 16:54:32,762 DEBUG [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/cf1 2023-02-07 16:54:32,763 INFO [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31438e1bba921e7d9b8bb86997119dbb columnFamilyName cf1 2023-02-07 16:54:32,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, ASSIGN in 329 msec 2023-02-07 16:54:32,763 INFO [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] regionserver.HStore(310): Store=31438e1bba921e7d9b8bb86997119dbb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:32,763 INFO [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,765 DEBUG [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/cf2 2023-02-07 16:54:32,765 DEBUG [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/cf2 2023-02-07 16:54:32,765 INFO [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31438e1bba921e7d9b8bb86997119dbb columnFamilyName cf2 2023-02-07 16:54:32,766 INFO [StoreOpener-31438e1bba921e7d9b8bb86997119dbb-1] regionserver.HStore(310): Store=31438e1bba921e7d9b8bb86997119dbb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:32,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:32,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:32,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 31438e1bba921e7d9b8bb86997119dbb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59810153, jitterRate=-0.10875926911830902}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:32,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 31438e1bba921e7d9b8bb86997119dbb: Running coprocessor pre-open hook at 1675788872759Writing region info on filesystem at 1675788872759Initializing all the Stores at 1675788872760 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788872760Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788872760Cleaning up temporary data from old regions at 1675788872768 (+8 ms)Cleaning up detritus from prior splits at 1675788872768Running coprocessor post-open hooks at 1675788872772 (+4 ms)Region opened successfully at 1675788872772 2023-02-07 16:54:32,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb., pid=61, masterSystemTime=1675788872741 2023-02-07 16:54:32,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:32,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:32,774 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=31438e1bba921e7d9b8bb86997119dbb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:32,775 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788872774"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788872774"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788872774"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788872774"}]},"ts":"1675788872774"} 2023-02-07 16:54:32,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-07 16:54:32,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 31438e1bba921e7d9b8bb86997119dbb, server=jenkins-hbase12.apache.org,41937,1675788846347 in 186 msec 2023-02-07 16:54:32,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-07 16:54:32,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, ASSIGN in 345 msec 2023-02-07 16:54:32,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 in 559 msec 2023-02-07 16:54:32,975 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:54:32,978 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:32,979 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:32,979 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:32,980 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:32,980 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:32,980 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-07 16:54:32,981 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionNoStoreFile 2023-02-07 16:54:32,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:32,984 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:32,984 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:32,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788872984"}]},"ts":"1675788872984"} 2023-02-07 16:54:32,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-07 16:54:32,986 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-07 16:54:33,053 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-07 16:54:33,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, UNASSIGN}] 2023-02-07 16:54:33,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, UNASSIGN 2023-02-07 16:54:33,055 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=1a6e4d9b038ccb4c495f43570894b413, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:33,056 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788873055"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788873055"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788873055"}]},"ts":"1675788873055"} 2023-02-07 16:54:33,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 1a6e4d9b038ccb4c495f43570894b413, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:33,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, UNASSIGN 2023-02-07 16:54:33,060 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=31438e1bba921e7d9b8bb86997119dbb, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:33,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788873060"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788873060"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788873060"}]},"ts":"1675788873060"} 2023-02-07 16:54:33,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 31438e1bba921e7d9b8bb86997119dbb, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:33,085 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,085 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-07 16:54:33,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:33,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 31438e1bba921e7d9b8bb86997119dbb, disabling compactions & flushes 2023-02-07 16:54:33,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:33,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:33,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. after waiting 0 ms 2023-02-07 16:54:33,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:33,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-07 16:54:33,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:33,215 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb. 2023-02-07 16:54:33,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 31438e1bba921e7d9b8bb86997119dbb: Waiting for close lock at 1675788873209Running coprocessor pre-close hooks at 1675788873209Disabling compacts and flushes for region at 1675788873209Disabling writes for close at 1675788873210 (+1 ms)Writing region close event to WAL at 1675788873210Running coprocessor post-close hooks at 1675788873214 (+4 ms)Closed at 1675788873215 (+1 ms) 2023-02-07 16:54:33,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:33,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:33,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1a6e4d9b038ccb4c495f43570894b413, disabling compactions & flushes 2023-02-07 16:54:33,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:33,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:33,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. after waiting 0 ms 2023-02-07 16:54:33,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:33,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=31438e1bba921e7d9b8bb86997119dbb, regionState=CLOSED 2023-02-07 16:54:33,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788873218"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788873218"}]},"ts":"1675788873218"} 2023-02-07 16:54:33,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-07 16:54:33,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-07 16:54:33,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 31438e1bba921e7d9b8bb86997119dbb, server=jenkins-hbase12.apache.org,41937,1675788846347 in 158 msec 2023-02-07 16:54:33,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:33,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413. 2023-02-07 16:54:33,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1a6e4d9b038ccb4c495f43570894b413: Waiting for close lock at 1675788873217Running coprocessor pre-close hooks at 1675788873217Disabling compacts and flushes for region at 1675788873217Disabling writes for close at 1675788873218 (+1 ms)Writing region close event to WAL at 1675788873218Running coprocessor post-close hooks at 1675788873222 (+4 ms)Closed at 1675788873222 2023-02-07 16:54:33,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=31438e1bba921e7d9b8bb86997119dbb, UNASSIGN in 168 msec 2023-02-07 16:54:33,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:33,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=1a6e4d9b038ccb4c495f43570894b413, regionState=CLOSED 2023-02-07 16:54:33,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788873224"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788873224"}]},"ts":"1675788873224"} 2023-02-07 16:54:33,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-07 16:54:33,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 1a6e4d9b038ccb4c495f43570894b413, server=jenkins-hbase12.apache.org,41937,1675788846347 in 169 msec 2023-02-07 16:54:33,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-07 16:54:33,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a6e4d9b038ccb4c495f43570894b413, UNASSIGN in 174 msec 2023-02-07 16:54:33,230 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788873230"}]},"ts":"1675788873230"} 2023-02-07 16:54:33,232 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-07 16:54:33,242 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-07 16:54:33,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 262 msec 2023-02-07 16:54:33,286 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,287 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-07 16:54:33,288 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-07 16:54:33,288 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,288 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,289 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,292 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,292 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,292 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,292 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-07 16:54:33,298 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:33,298 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:33,298 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:33,300 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/recovered.edits] 2023-02-07 16:54:33,301 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/recovered.edits] 2023-02-07 16:54:33,301 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/recovered.edits] 2023-02-07 16:54:33,308 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/recovered.edits/7.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb/recovered.edits/7.seqid 2023-02-07 16:54:33,308 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/recovered.edits/7.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413/recovered.edits/7.seqid 2023-02-07 16:54:33,308 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/recovered.edits/4.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d/recovered.edits/4.seqid 2023-02-07 16:54:33,309 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/31438e1bba921e7d9b8bb86997119dbb 2023-02-07 16:54:33,309 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:54:33,309 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionNoStoreFile/28b4cc61e5f5c6a50e245a62a87d6c2d 2023-02-07 16:54:33,309 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-07 16:54:33,312 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,314 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-07 16:54:33,316 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-07 16:54:33,318 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,318 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-07 16:54:33,318 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788873318"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:33,318 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788873318"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:33,318 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788873318"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:33,320 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:54:33,321 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 28b4cc61e5f5c6a50e245a62a87d6c2d, NAME => 'testSplitTableRegionNoStoreFile,,1675788871717.28b4cc61e5f5c6a50e245a62a87d6c2d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 31438e1bba921e7d9b8bb86997119dbb, NAME => 'testSplitTableRegionNoStoreFile,,1675788872220.31438e1bba921e7d9b8bb86997119dbb.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 1a6e4d9b038ccb4c495f43570894b413, NAME => 'testSplitTableRegionNoStoreFile,41,1675788872220.1a6e4d9b038ccb4c495f43570894b413.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 16:54:33,321 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-07 16:54:33,321 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788873321"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:33,323 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-07 16:54:33,328 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:54:33,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 39 msec 2023-02-07 16:54:33,393 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,393 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-07 16:54:33,395 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-07 16:54:33,407 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=408 (was 404) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-628429491_15 at /127.0.0.1:60604 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-628429491_15 at /127.0.0.1:38158 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 867), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=351 (was 351), ProcessCount=174 (was 174), AvailableMemoryMB=3048 (was 3054) 2023-02-07 16:54:33,421 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=174, AvailableMemoryMB=3047 2023-02-07 16:54:33,421 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:33,422 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,422 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,432 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:54:33,432 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:54:33,434 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,436 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:33,437 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:33,438 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,439 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 empty. 2023-02-07 16:54:33,439 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,439 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-07 16:54:33,448 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:33,449 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => bc78cd52db0f6344bb0f31fd08c63620, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:33,463 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:33,463 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing bc78cd52db0f6344bb0f31fd08c63620, disabling compactions & flushes 2023-02-07 16:54:33,463 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,463 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,463 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. after waiting 0 ms 2023-02-07 16:54:33,464 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,464 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,464 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for bc78cd52db0f6344bb0f31fd08c63620: Waiting for close lock at 1675788873463Disabling compacts and flushes for region at 1675788873463Disabling writes for close at 1675788873464 (+1 ms)Writing region close event to WAL at 1675788873464Closed at 1675788873464 2023-02-07 16:54:33,466 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:33,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675788873467"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788873467"}]},"ts":"1675788873467"} 2023-02-07 16:54:33,468 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:33,469 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:33,469 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788873469"}]},"ts":"1675788873469"} 2023-02-07 16:54:33,471 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-07 16:54:33,485 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:33,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:33,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:33,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:33,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:33,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, ASSIGN}] 2023-02-07 16:54:33,488 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, ASSIGN 2023-02-07 16:54:33,489 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=false 2023-02-07 16:54:33,639 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:33,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=bc78cd52db0f6344bb0f31fd08c63620, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:33,641 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675788873640"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788873640"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788873640"}]},"ts":"1675788873640"} 2023-02-07 16:54:33,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure bc78cd52db0f6344bb0f31fd08c63620, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:33,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bc78cd52db0f6344bb0f31fd08c63620, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:33,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:33,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:33,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,799 INFO [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,801 DEBUG [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1 2023-02-07 16:54:33,801 DEBUG [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1 2023-02-07 16:54:33,801 INFO [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bc78cd52db0f6344bb0f31fd08c63620 columnFamilyName cf1 2023-02-07 16:54:33,801 INFO [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] regionserver.HStore(310): Store=bc78cd52db0f6344bb0f31fd08c63620/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:33,802 INFO [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,803 DEBUG [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2 2023-02-07 16:54:33,803 DEBUG [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2 2023-02-07 16:54:33,804 INFO [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bc78cd52db0f6344bb0f31fd08c63620 columnFamilyName cf2 2023-02-07 16:54:33,804 INFO [StoreOpener-bc78cd52db0f6344bb0f31fd08c63620-1] regionserver.HStore(310): Store=bc78cd52db0f6344bb0f31fd08c63620/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:33,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:33,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:33,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:33,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened bc78cd52db0f6344bb0f31fd08c63620; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63638272, jitterRate=-0.051715850830078125}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:33,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for bc78cd52db0f6344bb0f31fd08c63620: Running coprocessor pre-open hook at 1675788873798Writing region info on filesystem at 1675788873798Initializing all the Stores at 1675788873799 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788873799Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788873799Cleaning up temporary data from old regions at 1675788873806 (+7 ms)Cleaning up detritus from prior splits at 1675788873807 (+1 ms)Running coprocessor post-open hooks at 1675788873812 (+5 ms)Region opened successfully at 1675788873812 2023-02-07 16:54:33,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620., pid=71, masterSystemTime=1675788873794 2023-02-07 16:54:33,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=bc78cd52db0f6344bb0f31fd08c63620, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:33,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675788873815"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788873815"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788873815"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788873815"}]},"ts":"1675788873815"} 2023-02-07 16:54:33,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-07 16:54:33,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure bc78cd52db0f6344bb0f31fd08c63620, server=jenkins-hbase12.apache.org,43011,1675788846300 in 173 msec 2023-02-07 16:54:33,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-07 16:54:33,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, ASSIGN in 331 msec 2023-02-07 16:54:33,820 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:33,820 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788873820"}]},"ts":"1675788873820"} 2023-02-07 16:54:33,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-07 16:54:33,857 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:33,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 425 msec 2023-02-07 16:54:33,942 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:33,942 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:33,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:33,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,944 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:33,944 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:33,944 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:33,944 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:33,944 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,945 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,958 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:33,958 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:33,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:33,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:33,958 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:33,958 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:33,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:33,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:33,959 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:33,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,960 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:33,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:33,960 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:33,960 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:33,960 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:33,961 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:54:33,968 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,979 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/4b95ba5a66af401f8b9af83eb52d616d 2023-02-07 16:54:33,979 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:33,979 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:33,979 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,000 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/f03ee9ce16ad449aa3633e5f36c97060 2023-02-07 16:54:34,005 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/4b95ba5a66af401f8b9af83eb52d616d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d 2023-02-07 16:54:34,011 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:34,011 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/f03ee9ce16ad449aa3633e5f36c97060 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060 2023-02-07 16:54:34,016 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:54:34,017 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 56ms, sequenceid=6, compaction requested=false 2023-02-07 16:54:34,018 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,018 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:34,018 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,020 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:34,020 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:34,020 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,020 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,020 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,032 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,032 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,032 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,032 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:34,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@190595b0[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:34,035 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:34,035 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,042 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,042 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,042 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,043 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,043 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,043 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:34,053 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,053 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,053 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,063 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,063 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,063 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,063 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,063 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,064 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,064 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:34,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:34,067 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:34,067 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:34,067 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:34,067 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,067 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,074 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,075 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,075 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,076 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,077 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,077 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,147 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,147 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,147 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:34,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,147 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,148 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,248 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:34,248 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:34,248 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:34,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:34,265 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:34,265 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:34,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:34,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:34,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,268 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,268 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,268 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:34,268 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:34,268 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,268 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,284 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,284 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,284 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,284 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,285 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,285 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:34,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,286 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,286 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,286 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:34,287 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:34,287 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,287 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:34,287 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:34,295 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,297 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/3d357b3b9e1d481ab93fc84afc00e7bd 2023-02-07 16:54:34,305 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,305 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,305 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,306 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,306 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,315 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/16ca50d044814079a0271e0f1042e6de 2023-02-07 16:54:34,320 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/3d357b3b9e1d481ab93fc84afc00e7bd as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd 2023-02-07 16:54:34,325 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:34,326 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/16ca50d044814079a0271e0f1042e6de as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de 2023-02-07 16:54:34,330 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:54:34,331 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 44ms, sequenceid=19, compaction requested=false 2023-02-07 16:54:34,331 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:34,331 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,331 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:34,332 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:34,332 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,332 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,332 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,337 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,337 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,337 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,338 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:34,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@190850bc[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:34,339 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:34,339 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,348 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:34,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,348 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:34,358 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,358 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,358 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,369 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,369 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,369 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,369 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,369 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,369 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,369 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:34,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:34,372 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:34,372 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:34,372 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:34,372 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,372 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:34,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,380 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,380 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,382 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,382 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,382 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,390 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,390 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:34,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,390 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:34,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,390 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,391 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,491 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:34,491 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:34,491 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:34,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:34,507 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:34,507 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:34,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:34,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:34,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,510 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,510 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,510 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:34,510 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:34,510 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,510 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,516 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,516 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,516 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,516 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,516 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,517 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,518 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:34,518 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,518 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,518 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,526 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/c0e634adf8024a989c94827edacb2541 2023-02-07 16:54:34,537 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/f7c221b87cbd49709743e0e0452ea07e 2023-02-07 16:54:34,541 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/c0e634adf8024a989c94827edacb2541 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541 2023-02-07 16:54:34,545 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:34,546 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/f7c221b87cbd49709743e0e0452ea07e as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e 2023-02-07 16:54:34,550 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:54:34,551 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 33ms, sequenceid=32, compaction requested=true 2023-02-07 16:54:34,551 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:34,551 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,551 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:34,551 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:34,551 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,551 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,551 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,561 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,579 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,579 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,579 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,590 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,590 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,590 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,590 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,590 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,590 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,593 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:34,593 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f29214f[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:34,593 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:34,593 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,600 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:34,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,601 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:34,601 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:34,611 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,611 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,611 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,611 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,611 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,611 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,621 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,621 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,621 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,621 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:34,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:34,625 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:34,625 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:34,625 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:34,625 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,625 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,633 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,633 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,634 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,635 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,635 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,642 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,642 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,642 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,643 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,642 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,643 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,643 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:34,643 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,642 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,642 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,643 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:34,643 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,643 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,643 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,644 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,744 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:34,744 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:34,744 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:34,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:34,757 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:34,757 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:34,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:34,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:34,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,759 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,759 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,759 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:34,759 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:34,760 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,760 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,769 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,769 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,769 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,769 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,769 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,769 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:34,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,770 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:34,770 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,771 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,770 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:34,771 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:34,771 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,771 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:34,771 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:34,779 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,779 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,779 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,779 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,779 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,779 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,782 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/3ab96609e5e04b8a8d7a7806eefa3988 2023-02-07 16:54:34,793 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/a759ca0df7284c5fae849807ba5a4eae 2023-02-07 16:54:34,796 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/3ab96609e5e04b8a8d7a7806eefa3988 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988 2023-02-07 16:54:34,800 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:34,800 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/a759ca0df7284c5fae849807ba5a4eae as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae 2023-02-07 16:54:34,804 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:54:34,805 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 34ms, sequenceid=45, compaction requested=true 2023-02-07 16:54:34,805 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:34,805 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:34,805 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:34,805 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:34,805 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,805 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:34,805 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:34,811 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,811 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,811 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,811 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:34,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:34,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b40a756[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:34,812 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:34,812 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,821 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,821 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,821 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,822 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,822 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:34,822 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:34,832 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,832 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,832 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,832 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,842 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,842 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,842 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,878 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:34,878 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,878 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:34,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:34,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:34,881 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:34,881 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:34,881 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:34,881 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,881 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:34,895 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,895 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,895 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:34,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:34,895 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,895 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,895 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,896 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,896 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:34,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:34,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:34,906 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:34,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:34,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:34,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:34,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:34,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:34,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:34,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:34,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:34,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,007 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:35,007 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,007 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:35,021 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,021 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,022 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:35,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:35,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,023 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,023 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,023 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:35,023 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:35,024 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,024 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,032 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,032 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,032 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,032 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,032 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,032 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:35,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:35,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:35,034 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,034 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:35,035 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:35,042 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,047 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/5781d97996ba4830a8b4105b13ac95d6 2023-02-07 16:54:35,053 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,053 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,053 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,053 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,053 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,059 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 2023-02-07 16:54:35,064 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/5781d97996ba4830a8b4105b13ac95d6 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6 2023-02-07 16:54:35,068 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:35,069 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 2023-02-07 16:54:35,073 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:54:35,074 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 39ms, sequenceid=58, compaction requested=true 2023-02-07 16:54:35,074 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:35,075 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,075 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:35,075 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:35,075 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,075 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,075 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,084 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,085 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,085 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:35,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e5d119b[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:35,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:35,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,095 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,096 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:35,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,095 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:35,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,096 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:35,105 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,106 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,105 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,106 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,106 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,116 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,116 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,116 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:35,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:35,119 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:35,119 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:35,119 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:35,119 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,119 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:35,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,127 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,128 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,129 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,129 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,130 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,163 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,164 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,163 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,163 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,164 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,164 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,164 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,164 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:35,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,164 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:35,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,164 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,165 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,165 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,165 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,165 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,165 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,265 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:35,265 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,265 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:35,280 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,280 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:35,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:35,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,282 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,282 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,282 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:35,282 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:35,283 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,283 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,295 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,295 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,295 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,295 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,296 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,296 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:35,297 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,297 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,297 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,297 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:54:35,305 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,306 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/a4884bfadd6c4a71af0475229a332708 2023-02-07 16:54:35,316 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,316 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,316 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,316 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,316 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,324 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/70eefc80ae0649cfa9d30e454ecaebd6 2023-02-07 16:54:35,328 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/a4884bfadd6c4a71af0475229a332708 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708 2023-02-07 16:54:35,355 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:35,356 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/70eefc80ae0649cfa9d30e454ecaebd6 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6 2023-02-07 16:54:35,360 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:54:35,361 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 64ms, sequenceid=71, compaction requested=true 2023-02-07 16:54:35,361 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:35,361 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,361 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:35,361 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:35,361 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,361 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,362 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,369 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,369 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,369 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,369 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:35,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@358b315c[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:35,371 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:35,371 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,379 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,379 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:35,380 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:35,390 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,390 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,390 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,400 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,400 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,400 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,400 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,401 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,400 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,401 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:35,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:35,403 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:35,403 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:35,403 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:35,403 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,403 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,411 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,412 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,412 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,414 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,414 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,414 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,421 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,421 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,421 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,421 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,421 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,421 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:35,422 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:35,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:35,422 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,523 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:35,523 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,523 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:35,536 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,536 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,537 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:35,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:35,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,539 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,539 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,539 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:35,539 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:35,539 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,539 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,548 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,548 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,548 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,548 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,548 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,548 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,549 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,549 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,549 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,550 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:35,550 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,550 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:35,550 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:35,550 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-07 16:54:35,558 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,558 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,558 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,558 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,558 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,558 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:35,558 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,561 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:35,569 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:35,572 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:35,573 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/42b7d5e83a294c74b779f74a4df646a0 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:35,576 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:35,576 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0, entries=9, sequenceid=83, filesize=5.1 K 2023-02-07 16:54:35,577 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/efc1d2d68d674dbeb72928cd441bd16c as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:35,580 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:35,581 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c, entries=9, sequenceid=83, filesize=5.1 K 2023-02-07 16:54:35,581 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 31ms, sequenceid=83, compaction requested=true 2023-02-07 16:54:35,581 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:35,581 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:35,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:35,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,582 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,590 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,590 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,590 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,590 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:35,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43de5dfb[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:35,592 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:35,592 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,601 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:35,601 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:35,638 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,638 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,638 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,653 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,653 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,653 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,653 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,653 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,653 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,653 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:35,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:35,657 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:35,657 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:35,657 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:35,657 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,657 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:35,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,664 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,664 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,666 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,666 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,666 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,674 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,674 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:35,674 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,674 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,775 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:35,775 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,775 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:35,784 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:35,784 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:35,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:35,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:35,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,786 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,786 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,786 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:35,786 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:35,786 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,790 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,790 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,790 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,790 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,790 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,790 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:35,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,791 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,791 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,791 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,791 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,791 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,792 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,792 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,792 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:35,792 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:35,792 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,792 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:35,792 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-07 16:54:35,800 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,801 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:35,804 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:35,811 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,811 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,811 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,811 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,811 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,812 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:35,815 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:35,816 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/77b89ab0438d446faf3310168d8276f2 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:35,819 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:35,820 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2, entries=5, sequenceid=91, filesize=5.0 K 2023-02-07 16:54:35,820 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/63a96fac6ca249309abf39ab88422633 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:35,824 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:35,824 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633, entries=5, sequenceid=91, filesize=5.0 K 2023-02-07 16:54:35,825 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 33ms, sequenceid=91, compaction requested=true 2023-02-07 16:54:35,825 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:35,825 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:35,825 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:35,825 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:35,825 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,825 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:35,825 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:35,832 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,832 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,832 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,832 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:35,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:35,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dfa28c1[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:35,834 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:35,834 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,842 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,842 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,842 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,843 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,843 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:35,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:35,843 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:35,853 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,853 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,853 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,853 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,853 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,853 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,853 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,863 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:35,864 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,864 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:35,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:35,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:35,866 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:35,866 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:35,867 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:35,867 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,867 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:35,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,875 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,874 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:35,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,875 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,875 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,876 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:35,877 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,877 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:35,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,913 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:35,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:35,914 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:35,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,913 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:35,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:35,914 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,014 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:36,014 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:36,014 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:36,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:36,022 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:36,022 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:36,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:54:36,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:54:36,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,024 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:36,024 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:36,024 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-07 16:54:36,024 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-07 16:54:36,025 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,025 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,042 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,042 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:36,042 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:36,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:36,042 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:36,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:36,043 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,043 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:54:36,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:36,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:36,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-07 16:54:36,045 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:54:36,045 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:54:36,045 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:36,045 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. started... 2023-02-07 16:54:36,045 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-07 16:54:36,053 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,054 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:36,058 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:36,063 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,063 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,063 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:36,064 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,064 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:36,065 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:36,068 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:36,069 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/96c9d184f77e478780a5806ebff3e0a5 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:36,072 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:36,073 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5, entries=5, sequenceid=99, filesize=5.0 K 2023-02-07 16:54:36,073 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/b63138cdc9c34c7b9d39bde10b5a536a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:36,077 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:36,077 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a, entries=5, sequenceid=99, filesize=5.0 K 2023-02-07 16:54:36,078 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 33ms, sequenceid=99, compaction requested=true 2023-02-07 16:54:36,078 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bc78cd52db0f6344bb0f31fd08c63620: 2023-02-07 16:54:36,078 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:36,078 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:54:36,078 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:54:36,078 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,078 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-07 16:54:36,078 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-07 16:54:36,084 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:36,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:36,085 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,085 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:54:36,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:36,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:36,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:36,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-07 16:54:36,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b2d232c[Count = 0] remaining members to acquire global barrier 2023-02-07 16:54:36,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-07 16:54:36,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:54:36,096 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:54:36,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:36,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:36,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:36,116 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:36,116 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:36,116 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,116 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-07 16:54:36,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:36,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:36,116 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:36,116 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:36,116 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-07 16:54:36,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:36,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:36,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:36,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:54:36,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:54:36,120 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-07 16:54:36,120 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:54:36,120 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:54:36,120 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,120 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:54:36,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:36,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:54:36,126 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:36,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:54:36,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:36,127 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:36,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:36,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,127 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:54:36,128 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:54:36,129 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,129 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,130 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:54:36,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:36,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,137 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-07 16:54:36,138 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:54:36,137 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:36,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:36,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:54:36,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:54:36,137 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:54:36,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:54:36,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:36,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:36,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:54:36,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:36,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,138 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:36,238 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:54:36,238 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:36,238 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:36,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:54:36,242 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,242 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:36,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:36,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:36,244 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:36,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, UNASSIGN}] 2023-02-07 16:54:36,246 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, UNASSIGN 2023-02-07 16:54:36,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=bc78cd52db0f6344bb0f31fd08c63620, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,246 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675788876246"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788876246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788876246"}]},"ts":"1675788876246"} 2023-02-07 16:54:36,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure bc78cd52db0f6344bb0f31fd08c63620, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:36,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing bc78cd52db0f6344bb0f31fd08c63620, disabling compactions & flushes 2023-02-07 16:54:36,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:36,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:36,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. after waiting 0 ms 2023-02-07 16:54:36,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:36,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing bc78cd52db0f6344bb0f31fd08c63620 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-07 16:54:36,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:36,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:36,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:36,425 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:36,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf1/c1c539d8d0614ed6b2abe59b81f226ec as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:36,429 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:36,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec, entries=1, sequenceid=103, filesize=4.9 K 2023-02-07 16:54:36,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.tmp/cf2/ca4938bf04ac4bd3a66a461989680368 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:36,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:36,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368, entries=1, sequenceid=103, filesize=4.9 K 2023-02-07 16:54:36,435 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 35ms, sequenceid=103, compaction requested=true 2023-02-07 16:54:36,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-07 16:54:36,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:36,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. 2023-02-07 16:54:36,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for bc78cd52db0f6344bb0f31fd08c63620: Waiting for close lock at 1675788876400Running coprocessor pre-close hooks at 1675788876400Disabling compacts and flushes for region at 1675788876400Disabling writes for close at 1675788876400Obtaining lock to block concurrent updates at 1675788876400Preparing flush snapshotting stores in bc78cd52db0f6344bb0f31fd08c63620 at 1675788876400Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675788876401 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620. at 1675788876401Flushing bc78cd52db0f6344bb0f31fd08c63620/cf1: creating writer at 1675788876402 (+1 ms)Flushing bc78cd52db0f6344bb0f31fd08c63620/cf1: appending metadata at 1675788876404 (+2 ms)Flushing bc78cd52db0f6344bb0f31fd08c63620/cf1: closing flushed file at 1675788876404Flushing bc78cd52db0f6344bb0f31fd08c63620/cf2: creating writer at 1675788876413 (+9 ms)Flushing bc78cd52db0f6344bb0f31fd08c63620/cf2: appending metadata at 1675788876415 (+2 ms)Flushing bc78cd52db0f6344bb0f31fd08c63620/cf2: closing flushed file at 1675788876415Flushing bc78cd52db0f6344bb0f31fd08c63620/cf1: reopening flushed file at 1675788876426 (+11 ms)Flushing bc78cd52db0f6344bb0f31fd08c63620/cf2: reopening flushed file at 1675788876431 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for bc78cd52db0f6344bb0f31fd08c63620 in 35ms, sequenceid=103, compaction requested=true at 1675788876435 (+4 ms)Writing region close event to WAL at 1675788876437 (+2 ms)Running coprocessor post-close hooks at 1675788876440 (+3 ms)Closed at 1675788876440 2023-02-07 16:54:36,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,442 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=bc78cd52db0f6344bb0f31fd08c63620, regionState=CLOSED 2023-02-07 16:54:36,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675788876442"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788876442"}]},"ts":"1675788876442"} 2023-02-07 16:54:36,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-07 16:54:36,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure bc78cd52db0f6344bb0f31fd08c63620, server=jenkins-hbase12.apache.org,43011,1675788846300 in 195 msec 2023-02-07 16:54:36,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-07 16:54:36,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bc78cd52db0f6344bb0f31fd08c63620, UNASSIGN in 200 msec 2023-02-07 16:54:36,457 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=bc78cd52db0f6344bb0f31fd08c63620, threads=16 2023-02-07 16:54:36,457 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,457 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,457 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,458 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,462 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,463 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,469 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,469 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,469 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,469 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,474 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:36,477 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0, top=false 2023-02-07 16:54:36,477 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae, top=false 2023-02-07 16:54:36,478 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:36,478 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368, top=true 2023-02-07 16:54:36,479 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:36,483 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988, top=false 2023-02-07 16:54:36,485 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:36,486 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:36,490 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6, top=false 2023-02-07 16:54:36,490 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:36,490 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a, top=true 2023-02-07 16:54:36,492 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:36,496 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6, top=false 2023-02-07 16:54:36,497 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:36,498 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:36,498 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2, top=true 2023-02-07 16:54:36,499 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:36,499 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633, top=true 2023-02-07 16:54:36,538 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988 for child: 79c8a3296cee557a5e6281665f3308bc, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,541 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,541 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,542 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633 for child: 856cb5044d0fb5e42dada9679874799f, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,542 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,542 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,543 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6 for child: 79c8a3296cee557a5e6281665f3308bc, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,543 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2 for child: 856cb5044d0fb5e42dada9679874799f, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,543 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,543 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,545 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:36,546 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708, top=false 2023-02-07 16:54:36,547 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,547 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,549 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0 for child: 79c8a3296cee557a5e6281665f3308bc, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,549 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:36,549 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5, top=true 2023-02-07 16:54:36,550 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:36,556 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6 for child: 79c8a3296cee557a5e6281665f3308bc, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,556 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a for child: 856cb5044d0fb5e42dada9679874799f, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,556 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae for child: 79c8a3296cee557a5e6281665f3308bc, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,556 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368 for child: 856cb5044d0fb5e42dada9679874799f, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,556 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,556 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,560 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,561 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,573 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:36,582 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5 for child: 856cb5044d0fb5e42dada9679874799f, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,582 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,582 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708 for child: 79c8a3296cee557a5e6281665f3308bc, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,586 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,606 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:36,606 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,606 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec, top=true 2023-02-07 16:54:36,631 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,631 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:36,642 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,643 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/.splits/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec for child: 856cb5044d0fb5e42dada9679874799f, parent: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,643 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,649 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,656 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,663 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,671 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,672 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,673 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0 for region: bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:36,673 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region bc78cd52db0f6344bb0f31fd08c63620 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:54:36,723 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-07 16:54:36,725 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-07 16:54:36,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675788876726"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675788876726"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675788876726"}]},"ts":"1675788876726"} 2023-02-07 16:54:36,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788876726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788876726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788876726"}]},"ts":"1675788876726"} 2023-02-07 16:54:36,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788876726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788876726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788876726"}]},"ts":"1675788876726"} 2023-02-07 16:54:36,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, ASSIGN}] 2023-02-07 16:54:36,731 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, ASSIGN 2023-02-07 16:54:36,732 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=false 2023-02-07 16:54:36,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, ASSIGN 2023-02-07 16:54:36,733 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=false 2023-02-07 16:54:36,882 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:54:36,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=79c8a3296cee557a5e6281665f3308bc, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788876884"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788876884"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788876884"}]},"ts":"1675788876884"} 2023-02-07 16:54:36,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 79c8a3296cee557a5e6281665f3308bc, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:36,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=856cb5044d0fb5e42dada9679874799f, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:36,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788876888"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788876888"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788876888"}]},"ts":"1675788876888"} 2023-02-07 16:54:36,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 856cb5044d0fb5e42dada9679874799f, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:37,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 856cb5044d0fb5e42dada9679874799f, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.', STARTKEY => '60', ENDKEY => ''} 2023-02-07 16:54:37,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:37,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:37,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,043 INFO [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,044 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1 2023-02-07 16:54:37,044 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1 2023-02-07 16:54:37,044 INFO [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 856cb5044d0fb5e42dada9679874799f columnFamilyName cf1 2023-02-07 16:54:37,058 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd-top 2023-02-07 16:54:37,062 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,062 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0-top 2023-02-07 16:54:37,067 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d-top 2023-02-07 16:54:37,071 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541-top 2023-02-07 16:54:37,075 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:37,075 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:37,080 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:37,080 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:37,083 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:37,084 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:37,084 INFO [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(310): Store=856cb5044d0fb5e42dada9679874799f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:37,084 INFO [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,085 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2 2023-02-07 16:54:37,085 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2 2023-02-07 16:54:37,085 INFO [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 856cb5044d0fb5e42dada9679874799f columnFamilyName cf2 2023-02-07 16:54:37,097 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de-top 2023-02-07 16:54:37,101 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,102 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c-top 2023-02-07 16:54:37,106 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060-top 2023-02-07 16:54:37,110 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e-top 2023-02-07 16:54:37,113 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:37,113 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:37,116 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:37,116 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:37,119 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:37,119 DEBUG [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:37,120 INFO [StoreOpener-856cb5044d0fb5e42dada9679874799f-1] regionserver.HStore(310): Store=856cb5044d0fb5e42dada9679874799f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:37,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:37,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 856cb5044d0fb5e42dada9679874799f; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73267372, jitterRate=0.09176892042160034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:37,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 856cb5044d0fb5e42dada9679874799f: Running coprocessor pre-open hook at 1675788877041Writing region info on filesystem at 1675788877041Initializing all the Stores at 1675788877042 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788877043 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788877043Cleaning up temporary data from old regions at 1675788877122 (+79 ms)Cleaning up detritus from prior splits at 1675788877122Running coprocessor post-open hooks at 1675788877124 (+2 ms)Region opened successfully at 1675788877124 2023-02-07 16:54:37,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f., pid=78, masterSystemTime=1675788877037 2023-02-07 16:54:37,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,126 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:37,126 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:37,126 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,126 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1912): 856cb5044d0fb5e42dada9679874799f/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:37,126 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,126 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 856cb5044d0fb5e42dada9679874799f/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:37,126 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1912): 856cb5044d0fb5e42dada9679874799f/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:37,126 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 856cb5044d0fb5e42dada9679874799f/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,127 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp, totalSize=34.8 K 2023-02-07 16:54:37,127 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp, totalSize=34.8 K 2023-02-07 16:54:37,127 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788873937 2023-02-07 16:54:37,127 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788873937 2023-02-07 16:54:37,127 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788874250 2023-02-07 16:54:37,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,128 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788874250 2023-02-07 16:54:37,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 79c8a3296cee557a5e6281665f3308bc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.', STARTKEY => '', ENDKEY => '60'} 2023-02-07 16:54:37,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:37,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,129 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788874493 2023-02-07 16:54:37,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:37,129 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788874493 2023-02-07 16:54:37,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=856cb5044d0fb5e42dada9679874799f, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:37,129 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675788875525 2023-02-07 16:54:37,129 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788877129"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788877129"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788877129"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788877129"}]},"ts":"1675788877129"} 2023-02-07 16:54:37,129 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675788875525 2023-02-07 16:54:37,129 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,129 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,130 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,130 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,130 INFO [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,130 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,130 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,132 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1 2023-02-07 16:54:37,132 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1 2023-02-07 16:54:37,133 INFO [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79c8a3296cee557a5e6281665f3308bc columnFamilyName cf1 2023-02-07 16:54:37,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-07 16:54:37,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 856cb5044d0fb5e42dada9679874799f, server=jenkins-hbase12.apache.org,43011,1675788846300 in 242 msec 2023-02-07 16:54:37,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, ASSIGN in 403 msec 2023-02-07 16:54:37,154 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd-bottom 2023-02-07 16:54:37,155 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 856cb5044d0fb5e42dada9679874799f#cf1#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,155 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] throttle.PressureAwareThroughputController(145): 856cb5044d0fb5e42dada9679874799f#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,159 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,160 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0-bottom 2023-02-07 16:54:37,164 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d-bottom 2023-02-07 16:54:37,168 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541-bottom 2023-02-07 16:54:37,172 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988 2023-02-07 16:54:37,175 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6 2023-02-07 16:54:37,178 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708 2023-02-07 16:54:37,178 INFO [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(310): Store=79c8a3296cee557a5e6281665f3308bc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:37,178 INFO [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,179 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2 2023-02-07 16:54:37,179 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2 2023-02-07 16:54:37,180 INFO [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79c8a3296cee557a5e6281665f3308bc columnFamilyName cf2 2023-02-07 16:54:37,183 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3f2cca5fe0ab4979a6f4ae4dd6c52e5a: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,183 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp/cf2/3f2cca5fe0ab4979a6f4ae4dd6c52e5a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/3f2cca5fe0ab4979a6f4ae4dd6c52e5a 2023-02-07 16:54:37,187 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3f2cca5fe0ab4979a6f4ae4dd6c52e5a: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,188 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 856cb5044d0fb5e42dada9679874799f/cf2 of 856cb5044d0fb5e42dada9679874799f into 3f2cca5fe0ab4979a6f4ae4dd6c52e5a(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,188 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 856cb5044d0fb5e42dada9679874799f: 2023-02-07 16:54:37,188 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f., storeName=856cb5044d0fb5e42dada9679874799f/cf2, priority=9, startTime=1675788877126; duration=0sec 2023-02-07 16:54:37,189 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,190 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8c1e4ad72c8c45d4b68b9e6f7fa1c8d9: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,191 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp/cf1/8c1e4ad72c8c45d4b68b9e6f7fa1c8d9 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/8c1e4ad72c8c45d4b68b9e6f7fa1c8d9 2023-02-07 16:54:37,193 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de-bottom 2023-02-07 16:54:37,194 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8c1e4ad72c8c45d4b68b9e6f7fa1c8d9: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,195 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 856cb5044d0fb5e42dada9679874799f/cf1 of 856cb5044d0fb5e42dada9679874799f into 8c1e4ad72c8c45d4b68b9e6f7fa1c8d9(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,195 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 856cb5044d0fb5e42dada9679874799f: 2023-02-07 16:54:37,195 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f., storeName=856cb5044d0fb5e42dada9679874799f/cf1, priority=9, startTime=1675788877125; duration=0sec 2023-02-07 16:54:37,195 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,196 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,196 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c-bottom 2023-02-07 16:54:37,199 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060-bottom 2023-02-07 16:54:37,202 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e-bottom 2023-02-07 16:54:37,206 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0 2023-02-07 16:54:37,213 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6 2023-02-07 16:54:37,217 DEBUG [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae 2023-02-07 16:54:37,217 INFO [StoreOpener-79c8a3296cee557a5e6281665f3308bc-1] regionserver.HStore(310): Store=79c8a3296cee557a5e6281665f3308bc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:37,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:54:37,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 79c8a3296cee557a5e6281665f3308bc; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61586011, jitterRate=-0.08229692280292511}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:54:37,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 79c8a3296cee557a5e6281665f3308bc: Running coprocessor pre-open hook at 1675788877129Writing region info on filesystem at 1675788877129Initializing all the Stores at 1675788877129Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788877129Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788877130 (+1 ms)Cleaning up temporary data from old regions at 1675788877219 (+89 ms)Cleaning up detritus from prior splits at 1675788877220 (+1 ms)Running coprocessor post-open hooks at 1675788877222 (+2 ms)Region opened successfully at 1675788877222 2023-02-07 16:54:37,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc., pid=77, masterSystemTime=1675788877037 2023-02-07 16:54:37,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,224 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:37,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:37,224 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:54:37,224 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,224 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1912): 79c8a3296cee557a5e6281665f3308bc/cf1 is initiating minor compaction (all files) 2023-02-07 16:54:37,224 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,224 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1912): 79c8a3296cee557a5e6281665f3308bc/cf2 is initiating minor compaction (all files) 2023-02-07 16:54:37,224 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 79c8a3296cee557a5e6281665f3308bc/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,224 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 79c8a3296cee557a5e6281665f3308bc/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,224 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/.tmp, totalSize=35.0 K 2023-02-07 16:54:37,224 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/.tmp, totalSize=35.0 K 2023-02-07 16:54:37,225 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788873937 2023-02-07 16:54:37,225 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788873937 2023-02-07 16:54:37,225 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788874250 2023-02-07 16:54:37,225 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788874250 2023-02-07 16:54:37,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788874493 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788874493 2023-02-07 16:54:37,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=79c8a3296cee557a5e6281665f3308bc, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788874745 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788874745 2023-02-07 16:54:37,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788877226"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788877226"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788877226"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788877226"}]},"ts":"1675788877226"} 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788875008 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788875008 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788875267 2023-02-07 16:54:37,226 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788875267 2023-02-07 16:54:37,227 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788875525 2023-02-07 16:54:37,227 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788875525 2023-02-07 16:54:37,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-07 16:54:37,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 79c8a3296cee557a5e6281665f3308bc, server=jenkins-hbase12.apache.org,43011,1675788846300 in 343 msec 2023-02-07 16:54:37,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-07 16:54:37,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, ASSIGN in 500 msec 2023-02-07 16:54:37,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f in 990 msec 2023-02-07 16:54:37,241 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] throttle.PressureAwareThroughputController(145): 79c8a3296cee557a5e6281665f3308bc#cf1#compaction#74 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,246 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79c8a3296cee557a5e6281665f3308bc#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,257 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/.tmp/cf1/6f7d265942974ecb918922081a7afc5d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/6f7d265942974ecb918922081a7afc5d 2023-02-07 16:54:37,261 DEBUG [Listener at localhost.localdomain/34435] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-07 16:54:37,263 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 79c8a3296cee557a5e6281665f3308bc/cf1 of 79c8a3296cee557a5e6281665f3308bc into 6f7d265942974ecb918922081a7afc5d(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,263 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79c8a3296cee557a5e6281665f3308bc: 2023-02-07 16:54:37,263 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc., storeName=79c8a3296cee557a5e6281665f3308bc/cf1, priority=9, startTime=1675788877223; duration=0sec 2023-02-07 16:54:37,263 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,267 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:38622, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-07 16:54:37,267 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/.tmp/cf2/33244ca2a08e4a58b57cbe0f0e1cefe0 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/33244ca2a08e4a58b57cbe0f0e1cefe0 2023-02-07 16:54:37,267 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HStore(1912): 79c8a3296cee557a5e6281665f3308bc/cf1 is initiating major compaction (all files) 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2edc7acd; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,269 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 79c8a3296cee557a5e6281665f3308bc/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,269 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/6f7d265942974ecb918922081a7afc5d] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/.tmp, totalSize=6.8 K 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-07 16:54:37,269 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] compactions.Compactor(207): Compacting 6f7d265942974ecb918922081a7afc5d, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788873937 2023-02-07 16:54:37,269 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. because compaction request was cancelled 2023-02-07 16:54:37,272 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 16:54:37,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.HStore(1912): 856cb5044d0fb5e42dada9679874799f/cf1 is initiating major compaction (all files) 2023-02-07 16:54:37,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@29427980; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:37,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-07 16:54:37,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.HStore(1912): 856cb5044d0fb5e42dada9679874799f/cf2 is initiating major compaction (all files) 2023-02-07 16:54:37,272 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@27218784; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-07 16:54:37,273 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 79c8a3296cee557a5e6281665f3308bc/cf2 of 79c8a3296cee557a5e6281665f3308bc into 33244ca2a08e4a58b57cbe0f0e1cefe0(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,273 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79c8a3296cee557a5e6281665f3308bc: 2023-02-07 16:54:37,273 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc., storeName=79c8a3296cee557a5e6281665f3308bc/cf2, priority=9, startTime=1675788877224; duration=0sec 2023-02-07 16:54:37,273 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-07 16:54:37,273 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 856cb5044d0fb5e42dada9679874799f/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,273 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/3f2cca5fe0ab4979a6f4ae4dd6c52e5a] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp, totalSize=5.0 K 2023-02-07 16:54:37,273 INFO [Listener at localhost.localdomain/34435] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-07 16:54:37,273 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 3f2cca5fe0ab4979a6f4ae4dd6c52e5a, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,281 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] throttle.PressureAwareThroughputController(145): 79c8a3296cee557a5e6281665f3308bc#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,282 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 856cb5044d0fb5e42dada9679874799f#cf2#compaction#77 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,295 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. is not mergeable because it has references 2023-02-07 16:54:37,295 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. is not mergeable because it has references 2023-02-07 16:54:37,297 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 08997c91ca7d49418d55e6710a3442db: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,297 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp/cf2/08997c91ca7d49418d55e6710a3442db as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/08997c91ca7d49418d55e6710a3442db 2023-02-07 16:54:37,298 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/.tmp/cf1/95e6d7bf2dd645498f7384989a516098 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/95e6d7bf2dd645498f7384989a516098 2023-02-07 16:54:37,301 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 08997c91ca7d49418d55e6710a3442db: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,302 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 79c8a3296cee557a5e6281665f3308bc/cf1 of 79c8a3296cee557a5e6281665f3308bc into 95e6d7bf2dd645498f7384989a516098(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,302 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 856cb5044d0fb5e42dada9679874799f/cf2 of 856cb5044d0fb5e42dada9679874799f into 08997c91ca7d49418d55e6710a3442db(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,302 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79c8a3296cee557a5e6281665f3308bc: 2023-02-07 16:54:37,302 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 856cb5044d0fb5e42dada9679874799f: 2023-02-07 16:54:37,304 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675788877272; duration=0sec 2023-02-07 16:54:37,304 INFO [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675788877269; duration=0sec 2023-02-07 16:54:37,304 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:37,304 DEBUG [RS:0;jenkins-hbase12:43011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:54:37,304 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 856cb5044d0fb5e42dada9679874799f/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,304 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/8c1e4ad72c8c45d4b68b9e6f7fa1c8d9] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp, totalSize=5.0 K 2023-02-07 16:54:37,305 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] compactions.Compactor(207): Compacting 8c1e4ad72c8c45d4b68b9e6f7fa1c8d9, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-07 16:54:37,308 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 856cb5044d0fb5e42dada9679874799f#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:54:37,321 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for abd6f28a2f6442d9afd7c2de61f21266: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,322 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/.tmp/cf1/abd6f28a2f6442d9afd7c2de61f21266 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/abd6f28a2f6442d9afd7c2de61f21266 2023-02-07 16:54:37,325 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for abd6f28a2f6442d9afd7c2de61f21266: NONE, but ROW specified in column family configuration 2023-02-07 16:54:37,326 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 856cb5044d0fb5e42dada9679874799f/cf1 of 856cb5044d0fb5e42dada9679874799f into abd6f28a2f6442d9afd7c2de61f21266(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:54:37,327 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 856cb5044d0fb5e42dada9679874799f: 2023-02-07 16:54:37,327 INFO [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675788877272; duration=0sec 2023-02-07 16:54:37,327 DEBUG [RS:0;jenkins-hbase12:43011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:54:37,405 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. is not mergeable because it has references 2023-02-07 16:54:37,405 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43011] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. is not mergeable because it has references 2023-02-07 16:54:37,413 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-07 16:54:37,414 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:37,415 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:37,415 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:37,416 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:37,416 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:37,416 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:37,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:37,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:37,420 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:37,420 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:37,420 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788877420"}]},"ts":"1675788877420"} 2023-02-07 16:54:37,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 16:54:37,421 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-07 16:54:37,484 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-07 16:54:37,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, UNASSIGN}] 2023-02-07 16:54:37,486 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, UNASSIGN 2023-02-07 16:54:37,487 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=856cb5044d0fb5e42dada9679874799f, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:37,487 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788877487"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788877487"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788877487"}]},"ts":"1675788877487"} 2023-02-07 16:54:37,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 856cb5044d0fb5e42dada9679874799f, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:37,490 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, UNASSIGN 2023-02-07 16:54:37,491 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=79c8a3296cee557a5e6281665f3308bc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:54:37,491 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788877491"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788877491"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788877491"}]},"ts":"1675788877491"} 2023-02-07 16:54:37,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 79c8a3296cee557a5e6281665f3308bc, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:54:37,521 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:37,521 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:37,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 16:54:37,584 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:54:37,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 856cb5044d0fb5e42dada9679874799f, disabling compactions & flushes 2023-02-07 16:54:37,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. after waiting 0 ms 2023-02-07 16:54:37,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,641 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/8c1e4ad72c8c45d4b68b9e6f7fa1c8d9, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec] to archive 2023-02-07 16:54:37,642 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:37,644 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,645 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,646 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,647 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,648 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:37,649 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:37,650 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/8c1e4ad72c8c45d4b68b9e6f7fa1c8d9 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/8c1e4ad72c8c45d4b68b9e6f7fa1c8d9 2023-02-07 16:54:37,651 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:37,653 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/3f2cca5fe0ab4979a6f4ae4dd6c52e5a, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368] to archive 2023-02-07 16:54:37,653 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:37,655 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,656 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,657 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,658 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,659 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:37,660 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:37,661 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/3f2cca5fe0ab4979a6f4ae4dd6c52e5a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/3f2cca5fe0ab4979a6f4ae4dd6c52e5a 2023-02-07 16:54:37,662 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:37,665 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-07 16:54:37,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:37,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f. 2023-02-07 16:54:37,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 856cb5044d0fb5e42dada9679874799f: Waiting for close lock at 1675788877640Running coprocessor pre-close hooks at 1675788877640Disabling compacts and flushes for region at 1675788877640Disabling writes for close at 1675788877641 (+1 ms)Writing region close event to WAL at 1675788877663 (+22 ms)Running coprocessor post-close hooks at 1675788877665 (+2 ms)Closed at 1675788877666 (+1 ms) 2023-02-07 16:54:37,668 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:37,668 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 79c8a3296cee557a5e6281665f3308bc, disabling compactions & flushes 2023-02-07 16:54:37,668 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. after waiting 0 ms 2023-02-07 16:54:37,668 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,668 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=856cb5044d0fb5e42dada9679874799f, regionState=CLOSED 2023-02-07 16:54:37,668 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788877668"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788877668"}]},"ts":"1675788877668"} 2023-02-07 16:54:37,669 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/6f7d265942974ecb918922081a7afc5d, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0-bottom] to archive 2023-02-07 16:54:37,670 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:37,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-07 16:54:37,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 856cb5044d0fb5e42dada9679874799f, server=jenkins-hbase12.apache.org,43011,1675788846300 in 182 msec 2023-02-07 16:54:37,671 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/4b95ba5a66af401f8b9af83eb52d616d.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=856cb5044d0fb5e42dada9679874799f, UNASSIGN in 186 msec 2023-02-07 16:54:37,673 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/3d357b3b9e1d481ab93fc84afc00e7bd.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,674 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/c0e634adf8024a989c94827edacb2541.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,675 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-3ab96609e5e04b8a8d7a7806eefa3988 2023-02-07 16:54:37,675 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-5781d97996ba4830a8b4105b13ac95d6 2023-02-07 16:54:37,676 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a4884bfadd6c4a71af0475229a332708 2023-02-07 16:54:37,677 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/6f7d265942974ecb918922081a7afc5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/6f7d265942974ecb918922081a7afc5d 2023-02-07 16:54:37,678 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/42b7d5e83a294c74b779f74a4df646a0.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,679 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c-bottom] to archive 2023-02-07 16:54:37,680 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:54:37,681 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f03ee9ce16ad449aa3633e5f36c97060.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,682 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/16ca50d044814079a0271e0f1042e6de.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,683 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/f7c221b87cbd49709743e0e0452ea07e.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,684 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-a759ca0df7284c5fae849807ba5a4eae 2023-02-07 16:54:37,685 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-54a7f2791a6a44e0b326bbed7ad80ac0 2023-02-07 16:54:37,686 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/testSplitTableRegionDeletedRowsDaughter=bc78cd52db0f6344bb0f31fd08c63620-70eefc80ae0649cfa9d30e454ecaebd6 2023-02-07 16:54:37,686 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/efc1d2d68d674dbeb72928cd441bd16c.bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:37,690 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/recovered.edits/112.seqid, newMaxSeqId=112, maxSeqId=106 2023-02-07 16:54:37,690 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:37,691 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc. 2023-02-07 16:54:37,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 79c8a3296cee557a5e6281665f3308bc: Waiting for close lock at 1675788877668Running coprocessor pre-close hooks at 1675788877668Disabling compacts and flushes for region at 1675788877668Disabling writes for close at 1675788877668Writing region close event to WAL at 1675788877687 (+19 ms)Running coprocessor post-close hooks at 1675788877690 (+3 ms)Closed at 1675788877691 (+1 ms) 2023-02-07 16:54:37,692 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:37,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=79c8a3296cee557a5e6281665f3308bc, regionState=CLOSED 2023-02-07 16:54:37,693 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675788877692"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788877692"}]},"ts":"1675788877692"} 2023-02-07 16:54:37,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-07 16:54:37,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 79c8a3296cee557a5e6281665f3308bc, server=jenkins-hbase12.apache.org,43011,1675788846300 in 202 msec 2023-02-07 16:54:37,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-07 16:54:37,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=79c8a3296cee557a5e6281665f3308bc, UNASSIGN in 210 msec 2023-02-07 16:54:37,697 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788877697"}]},"ts":"1675788877697"} 2023-02-07 16:54:37,698 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-07 16:54:37,722 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:37,722 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:37,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 16:54:37,726 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-07 16:54:37,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 310 msec 2023-02-07 16:54:38,023 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,024 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-07 16:54:38,025 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-07 16:54:38,025 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,025 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,029 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,029 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,029 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,029 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-07 16:54:38,033 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:38,033 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:38,033 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:38,035 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/recovered.edits] 2023-02-07 16:54:38,035 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/recovered.edits] 2023-02-07 16:54:38,035 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/recovered.edits] 2023-02-07 16:54:38,038 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/abd6f28a2f6442d9afd7c2de61f21266 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf1/abd6f28a2f6442d9afd7c2de61f21266 2023-02-07 16:54:38,038 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3ab96609e5e04b8a8d7a7806eefa3988 2023-02-07 16:54:38,038 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/95e6d7bf2dd645498f7384989a516098 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf1/95e6d7bf2dd645498f7384989a516098 2023-02-07 16:54:38,039 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/3d357b3b9e1d481ab93fc84afc00e7bd 2023-02-07 16:54:38,040 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/08997c91ca7d49418d55e6710a3442db to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/cf2/08997c91ca7d49418d55e6710a3442db 2023-02-07 16:54:38,040 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/33244ca2a08e4a58b57cbe0f0e1cefe0 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/cf2/33244ca2a08e4a58b57cbe0f0e1cefe0 2023-02-07 16:54:38,040 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/42b7d5e83a294c74b779f74a4df646a0 2023-02-07 16:54:38,042 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/4b95ba5a66af401f8b9af83eb52d616d 2023-02-07 16:54:38,042 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/recovered.edits/113.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f/recovered.edits/113.seqid 2023-02-07 16:54:38,042 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/recovered.edits/112.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc/recovered.edits/112.seqid 2023-02-07 16:54:38,043 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/5781d97996ba4830a8b4105b13ac95d6 2023-02-07 16:54:38,043 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/856cb5044d0fb5e42dada9679874799f 2023-02-07 16:54:38,043 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/79c8a3296cee557a5e6281665f3308bc 2023-02-07 16:54:38,044 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/77b89ab0438d446faf3310168d8276f2 2023-02-07 16:54:38,045 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/96c9d184f77e478780a5806ebff3e0a5 2023-02-07 16:54:38,045 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/a4884bfadd6c4a71af0475229a332708 2023-02-07 16:54:38,046 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c0e634adf8024a989c94827edacb2541 2023-02-07 16:54:38,047 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf1/c1c539d8d0614ed6b2abe59b81f226ec 2023-02-07 16:54:38,049 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/16ca50d044814079a0271e0f1042e6de 2023-02-07 16:54:38,050 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/54a7f2791a6a44e0b326bbed7ad80ac0 2023-02-07 16:54:38,051 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/63a96fac6ca249309abf39ab88422633 2023-02-07 16:54:38,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/70eefc80ae0649cfa9d30e454ecaebd6 2023-02-07 16:54:38,053 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/a759ca0df7284c5fae849807ba5a4eae 2023-02-07 16:54:38,053 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/b63138cdc9c34c7b9d39bde10b5a536a 2023-02-07 16:54:38,054 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/ca4938bf04ac4bd3a66a461989680368 2023-02-07 16:54:38,055 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/efc1d2d68d674dbeb72928cd441bd16c 2023-02-07 16:54:38,056 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f03ee9ce16ad449aa3633e5f36c97060 2023-02-07 16:54:38,057 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/cf2/f7c221b87cbd49709743e0e0452ea07e 2023-02-07 16:54:38,059 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/recovered.edits/106.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620/recovered.edits/106.seqid 2023-02-07 16:54:38,059 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bc78cd52db0f6344bb0f31fd08c63620 2023-02-07 16:54:38,059 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-07 16:54:38,061 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,063 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-07 16:54:38,065 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-07 16:54:38,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-07 16:54:38,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788878067"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:38,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788878067"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:38,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788878067"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:38,070 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:54:38,070 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => bc78cd52db0f6344bb0f31fd08c63620, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675788873433.bc78cd52db0f6344bb0f31fd08c63620.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 79c8a3296cee557a5e6281665f3308bc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675788876242.79c8a3296cee557a5e6281665f3308bc.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 856cb5044d0fb5e42dada9679874799f, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675788876242.856cb5044d0fb5e42dada9679874799f.', STARTKEY => '60', ENDKEY => ''}] 2023-02-07 16:54:38,070 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-07 16:54:38,070 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788878070"}]},"ts":"9223372036854775807"} 2023-02-07 16:54:38,072 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-07 16:54:38,117 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:54:38,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 91 msec 2023-02-07 16:54:38,130 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,130 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-07 16:54:38,131 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-07 16:54:38,145 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=413 (was 408) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46834 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60682 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46612 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46620 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:56676 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60752 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46596 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:57014 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:56998 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60726 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60684 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46642 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60210 [Waiting for operation #47] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60604 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:46628 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_288412932_15 at /127.0.0.1:60702 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa97c7a1-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3e7d9739-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=854 (was 867), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 351), ProcessCount=174 (was 174), AvailableMemoryMB=2980 (was 3047) 2023-02-07 16:54:38,158 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=413, OpenFileDescriptor=854, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=174, AvailableMemoryMB=2979 2023-02-07 16:54:38,159 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:54:38,159 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,159 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,169 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:54:38,169 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:54:38,171 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:54:38,172 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:54:38,173 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:54:38,175 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,175 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a empty. 2023-02-07 16:54:38,176 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,176 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-07 16:54:38,185 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:38,186 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7114752e270af1e078cd38e20cc2f45a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:54:38,195 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:38,196 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 7114752e270af1e078cd38e20cc2f45a, disabling compactions & flushes 2023-02-07 16:54:38,196 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,196 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,196 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. after waiting 0 ms 2023-02-07 16:54:38,196 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,196 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,196 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 7114752e270af1e078cd38e20cc2f45a: Waiting for close lock at 1675788878196Disabling compacts and flushes for region at 1675788878196Disabling writes for close at 1675788878196Writing region close event to WAL at 1675788878196Closed at 1675788878196 2023-02-07 16:54:38,199 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:54:38,200 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788878200"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788878200"}]},"ts":"1675788878200"} 2023-02-07 16:54:38,202 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:54:38,202 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:54:38,202 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788878202"}]},"ts":"1675788878202"} 2023-02-07 16:54:38,203 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-07 16:54:38,263 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:38,264 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:38,264 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:38,264 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:38,264 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:54:38,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, ASSIGN}] 2023-02-07 16:54:38,266 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, ASSIGN 2023-02-07 16:54:38,267 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:54:38,417 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:38,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:38,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788878419"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788878419"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788878419"}]},"ts":"1675788878419"} 2023-02-07 16:54:38,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:38,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7114752e270af1e078cd38e20cc2f45a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:38,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,577 INFO [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,578 DEBUG [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1 2023-02-07 16:54:38,578 DEBUG [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1 2023-02-07 16:54:38,579 INFO [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7114752e270af1e078cd38e20cc2f45a columnFamilyName cf1 2023-02-07 16:54:38,579 INFO [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] regionserver.HStore(310): Store=7114752e270af1e078cd38e20cc2f45a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:38,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:54:38,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 7114752e270af1e078cd38e20cc2f45a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67115433, jitterRate=9.788572788238525E-5}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 16:54:38,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 7114752e270af1e078cd38e20cc2f45a: Running coprocessor pre-open hook at 1675788878576Writing region info on filesystem at 1675788878576Initializing all the Stores at 1675788878577 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788878577Cleaning up temporary data from old regions at 1675788878581 (+4 ms)Cleaning up detritus from prior splits at 1675788878581Running coprocessor post-open hooks at 1675788878585 (+4 ms)Region opened successfully at 1675788878585 2023-02-07 16:54:38,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a., pid=87, masterSystemTime=1675788878572 2023-02-07 16:54:38,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:38,587 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788878587"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788878587"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788878587"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788878587"}]},"ts":"1675788878587"} 2023-02-07 16:54:38,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-07 16:54:38,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347 in 167 msec 2023-02-07 16:54:38,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-07 16:54:38,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, ASSIGN in 325 msec 2023-02-07 16:54:38,592 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:54:38,592 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788878592"}]},"ts":"1675788878592"} 2023-02-07 16:54:38,593 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-07 16:54:38,601 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:54:38,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 431 msec 2023-02-07 16:54:38,671 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,671 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,673 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:54:38,673 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:54:38,679 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$12(2441): Client=jenkins//136.243.104.168 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-07 16:54:38,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:54:38,696 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-07 16:54:38,697 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-07 16:54:38,697 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-07 16:54:38,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-07 16:54:38,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, REOPEN/MOVE}] 2023-02-07 16:54:38,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, REOPEN/MOVE 2023-02-07 16:54:38,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:38,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788878713"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788878713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788878713"}]},"ts":"1675788878713"} 2023-02-07 16:54:38,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:38,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 7114752e270af1e078cd38e20cc2f45a, disabling compactions & flushes 2023-02-07 16:54:38,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. after waiting 0 ms 2023-02-07 16:54:38,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-07 16:54:38,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:54:38,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:38,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 7114752e270af1e078cd38e20cc2f45a: Waiting for close lock at 1675788878867Running coprocessor pre-close hooks at 1675788878867Disabling compacts and flushes for region at 1675788878867Disabling writes for close at 1675788878868 (+1 ms)Writing region close event to WAL at 1675788878868Running coprocessor post-close hooks at 1675788878872 (+4 ms)Closed at 1675788878872 2023-02-07 16:54:38,872 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 7114752e270af1e078cd38e20cc2f45a to self. 2023-02-07 16:54:38,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:38,874 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=CLOSED 2023-02-07 16:54:38,874 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788878874"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788878874"}]},"ts":"1675788878874"} 2023-02-07 16:54:38,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-07 16:54:38,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347 in 160 msec 2023-02-07 16:54:38,877 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=true 2023-02-07 16:54:39,027 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:54:39,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:39,028 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788879027"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788879027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788879027"}]},"ts":"1675788879027"} 2023-02-07 16:54:39,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:54:39,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:39,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7114752e270af1e078cd38e20cc2f45a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:54:39,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:39,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:39,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,186 INFO [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,187 DEBUG [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1 2023-02-07 16:54:39,187 DEBUG [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1 2023-02-07 16:54:39,187 INFO [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7114752e270af1e078cd38e20cc2f45a columnFamilyName cf1 2023-02-07 16:54:39,187 INFO [StoreOpener-7114752e270af1e078cd38e20cc2f45a-1] regionserver.HStore(310): Store=7114752e270af1e078cd38e20cc2f45a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:54:39,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:54:39,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 7114752e270af1e078cd38e20cc2f45a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74572272, jitterRate=0.11121344566345215}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 16:54:39,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 7114752e270af1e078cd38e20cc2f45a: Running coprocessor pre-open hook at 1675788879185Writing region info on filesystem at 1675788879185Initializing all the Stores at 1675788879185Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788879186 (+1 ms)Cleaning up temporary data from old regions at 1675788879190 (+4 ms)Cleaning up detritus from prior splits at 1675788879190Running coprocessor post-open hooks at 1675788879193 (+3 ms)Region opened successfully at 1675788879193 2023-02-07 16:54:39,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a., pid=92, masterSystemTime=1675788879181 2023-02-07 16:54:39,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:39,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:54:39,195 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:54:39,195 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788879195"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788879195"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788879195"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788879195"}]},"ts":"1675788879195"} 2023-02-07 16:54:39,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-07 16:54:39,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347 in 167 msec 2023-02-07 16:54:39,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-07 16:54:39,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, REOPEN/MOVE in 485 msec 2023-02-07 16:54:39,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-07 16:54:39,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-02-07 16:54:39,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, ASSIGN}] 2023-02-07 16:54:39,204 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, ASSIGN 2023-02-07 16:54:39,205 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-07 16:54:39,355 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:54:39,356 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:54:39,356 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:54:39,356 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:54:39,356 DEBUG [jenkins-hbase12:39383] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-07 16:54:39,358 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=38a4d25035f0c2227e54aee84fa910d5, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:54:39,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788879357"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1675788879357"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675788879357"}]},"ts":"1675788879357"} 2023-02-07 16:54:39,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 38a4d25035f0c2227e54aee84fa910d5, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:54:39,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:54:39,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 38a4d25035f0c2227e54aee84fa910d5, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-07 16:54:39,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:54:39,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 38a4d25035f0c2227e54aee84fa910d5 2023-02-07 16:54:39,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:54:39,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 38a4d25035f0c2227e54aee84fa910d5 2023-02-07 16:54:39,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 38a4d25035f0c2227e54aee84fa910d5 2023-02-07 16:54:39,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:39,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:39,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:39,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:39,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:40,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:41,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,543 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:54:42,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:42,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:43,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,433 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-07 16:54:44,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:44,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:45,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:46,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:47,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:48,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:49,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:50,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:51,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:52,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:53,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:54,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:55,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:56,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:57,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:58,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:54:59,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:00,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:01,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:02,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:03,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:04,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:05,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:06,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,972 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 16:55:07,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:07,985 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 16:55:08,006 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-07 16:55:08,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:08,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:09,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:10,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:11,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:12,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:13,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:14,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:15,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:16,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:17,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:18,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:19,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:20,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:21,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:22,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:23,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:24,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:25,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:26,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:27,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:28,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:29,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:30,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:31,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:32,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:33,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:34,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:35,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:36,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:37,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,794 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:38,794 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:38,800 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:38,800 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:38,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 16:55:38,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:38,902 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:38,902 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:38,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 16:55:38,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,105 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:39,105 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:39,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 16:55:39,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,406 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:39,407 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:39,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 16:55:39,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:39,908 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:39,908 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:39,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-07 16:55:39,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:40,912 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:40,913 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:40,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:40,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:40,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:40,917 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 7114752e270af1e078cd38e20cc2f45a, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:40,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e exec-time=8 msec 2023-02-07 16:55:40,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:41,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:41,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-07 16:55:41,258 INFO [StoreOpener-38a4d25035f0c2227e54aee84fa910d5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 38a4d25035f0c2227e54aee84fa910d5 2023-02-07 16:55:41,266 DEBUG [StoreOpener-38a4d25035f0c2227e54aee84fa910d5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1 2023-02-07 16:55:41,266 DEBUG [StoreOpener-38a4d25035f0c2227e54aee84fa910d5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1 2023-02-07 16:55:41,267 INFO [StoreOpener-38a4d25035f0c2227e54aee84fa910d5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38a4d25035f0c2227e54aee84fa910d5 columnFamilyName cf1 2023-02-07 16:55:41,270 INFO [StoreOpener-38a4d25035f0c2227e54aee84fa910d5-1] regionserver.HStore(310): Store=38a4d25035f0c2227e54aee84fa910d5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:41,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 38a4d25035f0c2227e54aee84fa910d5; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69179361, jitterRate=0.03085280954837799}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-07 16:55:41,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 38a4d25035f0c2227e54aee84fa910d5: Running coprocessor pre-open hook at 1675788879518Initializing all the Stores at 1675788941257 (+61739 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788941257Running coprocessor post-open hooks at 1675788941274 (+17 ms)Region opened successfully at 1675788941274 2023-02-07 16:55:41,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5., pid=94, masterSystemTime=1675788879513 2023-02-07 16:55:41,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:55:41,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:55:41,277 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=38a4d25035f0c2227e54aee84fa910d5, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:41,277 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788941276"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1675788941276"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675788941276"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675788941276"}]},"ts":"1675788941276"} 2023-02-07 16:55:41,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-07 16:55:41,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 38a4d25035f0c2227e54aee84fa910d5, server=jenkins-hbase12.apache.org,40593,1675788846388 in 1 mins, 1.919 sec 2023-02-07 16:55:41,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-07 16:55:41,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, ASSIGN in 1 mins, 2.077 sec 2023-02-07 16:55:41,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.602 sec 2023-02-07 16:55:41,284 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675788878677","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62607,"client":"136.243.104.168:51346","queuetimems":0,"class":"HMaster"} 2023-02-07 16:55:41,285 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 609, skipping over this response of 3 bytes 2023-02-07 16:55:43,164 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up 2023-02-07 16:55:43,166 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:43,166 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,166 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,168 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,168 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,168 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,169 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,172 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,172 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 16:55:43,174 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788943173"}]},"ts":"1675788943173"} 2023-02-07 16:55:43,175 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-07 16:55:43,273 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,274 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 16:55:43,321 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-07 16:55:43,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, UNASSIGN}] 2023-02-07 16:55:43,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, UNASSIGN 2023-02-07 16:55:43,323 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=38a4d25035f0c2227e54aee84fa910d5, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:43,323 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788943323"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1675788943323"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675788943323"}]},"ts":"1675788943323"} 2023-02-07 16:55:43,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 38a4d25035f0c2227e54aee84fa910d5, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:43,327 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, UNASSIGN 2023-02-07 16:55:43,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:43,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788943329"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788943329"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788943329"}]},"ts":"1675788943329"} 2023-02-07 16:55:43,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:43,475 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,475 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 16:55:43,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 38a4d25035f0c2227e54aee84fa910d5 2023-02-07 16:55:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 38a4d25035f0c2227e54aee84fa910d5, disabling compactions & flushes 2023-02-07 16:55:43,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:55:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:55:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. after waiting 0 ms 2023-02-07 16:55:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:55:43,481 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:43,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5. 2023-02-07 16:55:43,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 38a4d25035f0c2227e54aee84fa910d5: Waiting for close lock at 1675788943477Running coprocessor pre-close hooks at 1675788943477Disabling compacts and flushes for region at 1675788943477Disabling writes for close at 1675788943478 (+1 ms)Writing region close event to WAL at 1675788943481 (+3 ms)Running coprocessor post-close hooks at 1675788943481Closed at 1675788943482 (+1 ms) 2023-02-07 16:55:43,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 38a4d25035f0c2227e54aee84fa910d5 2023-02-07 16:55:43,485 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=38a4d25035f0c2227e54aee84fa910d5, regionState=CLOSED 2023-02-07 16:55:43,485 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788943485"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675788943485"}]},"ts":"1675788943485"} 2023-02-07 16:55:43,487 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:55:43,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 7114752e270af1e078cd38e20cc2f45a, disabling compactions & flushes 2023-02-07 16:55:43,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:55:43,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:55:43,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. after waiting 0 ms 2023-02-07 16:55:43,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:55:43,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-07 16:55:43,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 38a4d25035f0c2227e54aee84fa910d5, server=jenkins-hbase12.apache.org,40593,1675788846388 in 161 msec 2023-02-07 16:55:43,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-07 16:55:43,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:43,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a. 2023-02-07 16:55:43,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 7114752e270af1e078cd38e20cc2f45a: Waiting for close lock at 1675788943487Running coprocessor pre-close hooks at 1675788943487Disabling compacts and flushes for region at 1675788943487Disabling writes for close at 1675788943488 (+1 ms)Writing region close event to WAL at 1675788943489 (+1 ms)Running coprocessor post-close hooks at 1675788943494 (+5 ms)Closed at 1675788943495 (+1 ms) 2023-02-07 16:55:43,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5, UNASSIGN in 171 msec 2023-02-07 16:55:43,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:55:43,498 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=7114752e270af1e078cd38e20cc2f45a, regionState=CLOSED 2023-02-07 16:55:43,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675788943498"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788943498"}]},"ts":"1675788943498"} 2023-02-07 16:55:43,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-07 16:55:43,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 7114752e270af1e078cd38e20cc2f45a, server=jenkins-hbase12.apache.org,41937,1675788846347 in 168 msec 2023-02-07 16:55:43,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-07 16:55:43,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=7114752e270af1e078cd38e20cc2f45a, UNASSIGN in 180 msec 2023-02-07 16:55:43,505 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788943505"}]},"ts":"1675788943505"} 2023-02-07 16:55:43,506 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-07 16:55:43,512 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-07 16:55:43,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 344 msec 2023-02-07 16:55:43,776 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,776 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-07 16:55:43,778 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-07 16:55:43,778 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,778 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,779 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,783 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,783 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,783 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,784 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-07 16:55:43,790 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:55:43,792 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/recovered.edits] 2023-02-07 16:55:43,798 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/recovered.edits/7.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a/recovered.edits/7.seqid 2023-02-07 16:55:43,798 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/7114752e270af1e078cd38e20cc2f45a 2023-02-07 16:55:43,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-07 16:55:43,800 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,802 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-07 16:55:43,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-07 16:55:43,804 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,804 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-07 16:55:43,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788943804"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:43,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788943804"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:43,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-07 16:55:43,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7114752e270af1e078cd38e20cc2f45a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675788878170.7114752e270af1e078cd38e20cc2f45a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 38a4d25035f0c2227e54aee84fa910d5, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675788878170_0001.38a4d25035f0c2227e54aee84fa910d5.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-07 16:55:43,806 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-07 16:55:43,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788943806"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:43,807 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-07 16:55:43,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:43,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 39 msec 2023-02-07 16:55:43,886 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,886 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-07 16:55:43,888 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-07 16:55:43,909 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 413), OpenFileDescriptor=658 (was 854), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=308 (was 339), ProcessCount=171 (was 174), AvailableMemoryMB=3018 (was 2979) - AvailableMemoryMB LEAK? - 2023-02-07 16:55:43,923 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=171, AvailableMemoryMB=3017 2023-02-07 16:55:43,924 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:43,924 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:43,924 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:43,965 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:55:43,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:55:43,968 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:43,971 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:55:43,973 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:55:43,975 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:43,976 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 empty. 2023-02-07 16:55:43,977 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:43,977 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-07 16:55:43,993 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-07 16:55:43,994 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => a5e454f361915b4710ba3b438c9f0076, NAME => 'testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:55:44,006 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:44,006 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing a5e454f361915b4710ba3b438c9f0076, disabling compactions & flushes 2023-02-07 16:55:44,006 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,006 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,006 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. after waiting 0 ms 2023-02-07 16:55:44,006 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,006 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,007 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for a5e454f361915b4710ba3b438c9f0076: Waiting for close lock at 1675788944006Disabling compacts and flushes for region at 1675788944006Disabling writes for close at 1675788944006Writing region close event to WAL at 1675788944006Closed at 1675788944006 2023-02-07 16:55:44,009 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:55:44,010 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675788944010"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788944010"}]},"ts":"1675788944010"} 2023-02-07 16:55:44,012 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:55:44,012 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:55:44,012 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788944012"}]},"ts":"1675788944012"} 2023-02-07 16:55:44,013 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-07 16:55:44,177 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:55:44,180 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:55:44,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:55:44,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:55:44,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:55:44,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, ASSIGN}] 2023-02-07 16:55:44,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, ASSIGN 2023-02-07 16:55:44,187 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,43011,1675788846300; forceNewPlan=false, retain=false 2023-02-07 16:55:44,337 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:55:44,338 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=a5e454f361915b4710ba3b438c9f0076, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675788944338"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788944338"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788944338"}]},"ts":"1675788944338"} 2023-02-07 16:55:44,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure a5e454f361915b4710ba3b438c9f0076, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:55:44,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a5e454f361915b4710ba3b438c9f0076, NAME => 'testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:55:44,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:44,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:44,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,505 INFO [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,508 DEBUG [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1 2023-02-07 16:55:44,508 DEBUG [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1 2023-02-07 16:55:44,509 INFO [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5e454f361915b4710ba3b438c9f0076 columnFamilyName cf1 2023-02-07 16:55:44,510 INFO [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] regionserver.HStore(310): Store=a5e454f361915b4710ba3b438c9f0076/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:44,510 INFO [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,512 DEBUG [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2 2023-02-07 16:55:44,512 DEBUG [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2 2023-02-07 16:55:44,512 INFO [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5e454f361915b4710ba3b438c9f0076 columnFamilyName cf2 2023-02-07 16:55:44,513 INFO [StoreOpener-a5e454f361915b4710ba3b438c9f0076-1] regionserver.HStore(310): Store=a5e454f361915b4710ba3b438c9f0076/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:44,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:44,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:44,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:55:44,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened a5e454f361915b4710ba3b438c9f0076; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69137060, jitterRate=0.03022247552871704}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:44,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for a5e454f361915b4710ba3b438c9f0076: Running coprocessor pre-open hook at 1675788944503Writing region info on filesystem at 1675788944503Initializing all the Stores at 1675788944504 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788944504Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788944505 (+1 ms)Cleaning up temporary data from old regions at 1675788944516 (+11 ms)Cleaning up detritus from prior splits at 1675788944516Running coprocessor post-open hooks at 1675788944522 (+6 ms)Region opened successfully at 1675788944522 2023-02-07 16:55:44,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076., pid=104, masterSystemTime=1675788944493 2023-02-07 16:55:44,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=a5e454f361915b4710ba3b438c9f0076, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,526 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675788944526"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788944526"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788944526"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788944526"}]},"ts":"1675788944526"} 2023-02-07 16:55:44,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-07 16:55:44,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure a5e454f361915b4710ba3b438c9f0076, server=jenkins-hbase12.apache.org,43011,1675788846300 in 187 msec 2023-02-07 16:55:44,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-07 16:55:44,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, ASSIGN in 348 msec 2023-02-07 16:55:44,530 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:55:44,530 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788944530"}]},"ts":"1675788944530"} 2023-02-07 16:55:44,531 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-07 16:55:44,683 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:55:44,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 716 msec 2023-02-07 16:55:44,727 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:44,727 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:44,728 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:44,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 16:55:44,730 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 16:55:44,730 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:44,730 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 16:55:44,730 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 16:55:44,731 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,731 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,742 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,742 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:44,742 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:44,742 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:44,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:44,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:44,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:44,742 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:44,743 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:44,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,744 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:44,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:44,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:44,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:44,744 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:44,744 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,744 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. started... 2023-02-07 16:55:44,745 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:55:44,754 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:44,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:44,757 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/0a2856deedd443c195c8c6a365d5c29f 2023-02-07 16:55:44,772 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/13e4a6b61ea949c085b4b7325d53d556 2023-02-07 16:55:44,776 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/0a2856deedd443c195c8c6a365d5c29f as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/0a2856deedd443c195c8c6a365d5c29f 2023-02-07 16:55:44,779 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/0a2856deedd443c195c8c6a365d5c29f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:55:44,780 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/13e4a6b61ea949c085b4b7325d53d556 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/13e4a6b61ea949c085b4b7325d53d556 2023-02-07 16:55:44,784 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/13e4a6b61ea949c085b4b7325d53d556, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:55:44,785 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 40ms, sequenceid=6, compaction requested=false 2023-02-07 16:55:44,785 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-07 16:55:44,785 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:44,785 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:44,785 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:44,785 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:44,785 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,785 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:44,785 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:44,796 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,796 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:44,797 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,797 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:44,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:44,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 16:55:44,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77c0e0c4[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:44,798 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 16:55:44,799 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:44,807 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,807 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:44,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,807 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:44,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:44,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:44,808 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:44,818 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:44,818 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:44,818 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:44,828 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:44,828 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:44,828 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:44,839 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:44,839 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,839 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:44,839 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:44,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:44,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:44,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:55:44,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 16:55:44,842 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 16:55:44,842 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:44,842 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:44,842 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 16:55:44,842 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:44,850 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:44,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:44,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,850 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,851 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,852 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:44,852 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:44,852 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:44,860 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:44,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 16:55:44,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:44,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:44,861 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:44,860 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:44,861 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:44,861 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:44,861 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:44,861 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:44,861 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,961 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:44,961 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:44,961 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:44,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:44,983 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:44,983 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:44,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:44,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:44,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 16:55:44,986 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 16:55:44,986 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:44,986 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 16:55:44,986 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 16:55:44,986 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:44,986 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,029 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,029 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,029 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,029 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,030 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,031 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,033 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,035 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,034 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,035 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,035 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,035 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:45,035 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:45,035 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:45,036 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. started... 2023-02-07 16:55:45,036 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:45,047 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/c1580962b5904ffd892bc7df5edd99bf 2023-02-07 16:55:45,049 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,060 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,060 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,060 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,060 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,060 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,069 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/9208d4928fab48faa418fc64c3326b45 2023-02-07 16:55:45,073 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/c1580962b5904ffd892bc7df5edd99bf as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/c1580962b5904ffd892bc7df5edd99bf 2023-02-07 16:55:45,078 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/c1580962b5904ffd892bc7df5edd99bf, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:55:45,079 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/9208d4928fab48faa418fc64c3326b45 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/9208d4928fab48faa418fc64c3326b45 2023-02-07 16:55:45,083 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/9208d4928fab48faa418fc64c3326b45, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:55:45,084 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 48ms, sequenceid=19, compaction requested=false 2023-02-07 16:55:45,084 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:45,084 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:45,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:45,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:45,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,084 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,091 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,091 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:45,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:45,092 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,092 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:45,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:45,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:45,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 16:55:45,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46c05a42[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:45,095 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 16:55:45,095 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:45,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:45,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,102 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:45,102 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:45,102 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:45,103 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:45,103 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:45,122 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:45,122 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,122 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:45,133 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:45,133 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,133 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,133 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:45,134 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:45,134 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:45,134 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:55:45,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 16:55:45,138 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 16:55:45,138 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:45,138 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:45,139 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 16:55:45,139 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,144 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,145 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:45,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,145 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:45,146 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:45,147 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,148 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,148 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,154 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,154 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,155 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,154 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,154 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 16:55:45,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:45,155 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,156 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,156 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,156 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,256 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:45,256 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:45,256 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:45,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:45,269 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:45,269 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:45,269 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:45,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:45,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 16:55:45,271 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 16:55:45,271 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,271 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 16:55:45,271 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 16:55:45,272 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,272 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,281 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,281 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,281 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,281 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,281 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,281 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,282 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,282 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,282 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,282 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,282 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,282 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,283 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:45,283 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,283 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. started... 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,283 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,283 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:45,293 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,302 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,302 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,302 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,302 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,302 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,693 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/ebaf8afa801743c78d24db641602e7a9 2023-02-07 16:55:45,709 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/323c5f62e3f24f7eae57557dafb22687 2023-02-07 16:55:45,713 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/ebaf8afa801743c78d24db641602e7a9 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/ebaf8afa801743c78d24db641602e7a9 2023-02-07 16:55:45,716 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/ebaf8afa801743c78d24db641602e7a9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:55:45,717 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/323c5f62e3f24f7eae57557dafb22687 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/323c5f62e3f24f7eae57557dafb22687 2023-02-07 16:55:45,721 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/323c5f62e3f24f7eae57557dafb22687, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:55:45,722 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 439ms, sequenceid=32, compaction requested=true 2023-02-07 16:55:45,722 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:45,722 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:45,722 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:45,722 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:45,723 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,723 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,723 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,742 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,742 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:45,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:45,743 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:45,743 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:45,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 16:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a30ce7b[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:45,744 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 16:55:45,744 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,754 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,754 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,754 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:45,755 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,755 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:45,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:45,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:45,755 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:45,765 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:45,765 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,765 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:45,775 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:45,776 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,775 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:45,776 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,776 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,776 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:45,776 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:45,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:45,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:55:45,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 16:55:45,779 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 16:55:45,779 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:45,779 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:45,779 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 16:55:45,779 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:45,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:45,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:45,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:45,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,788 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:45,789 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,789 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,789 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,824 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,824 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,825 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,824 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,824 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 16:55:45,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:45,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:45,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:45,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:45,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:45,825 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:45,826 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,826 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,826 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:45,826 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,826 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,926 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:45,926 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:45,926 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:45,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:45,936 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:45,936 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:45,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:45,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:45,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 16:55:45,938 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 16:55:45,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 16:55:45,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 16:55:45,939 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,939 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,954 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,954 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,954 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,954 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:45,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:45,955 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,955 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,957 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:45,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,957 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:45,957 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:45,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:45,957 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:45,958 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:45,958 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:45,958 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:45,958 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. started... 2023-02-07 16:55:45,958 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:45,965 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,967 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/16ea0099906f49c8bbe2084a124f9a44 2023-02-07 16:55:45,975 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,975 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,976 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,976 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:45,976 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:45,979 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/fe4cb55689c14489bf281b808521e64d 2023-02-07 16:55:45,984 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/16ea0099906f49c8bbe2084a124f9a44 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/16ea0099906f49c8bbe2084a124f9a44 2023-02-07 16:55:45,988 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/16ea0099906f49c8bbe2084a124f9a44, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:55:45,989 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/fe4cb55689c14489bf281b808521e64d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/fe4cb55689c14489bf281b808521e64d 2023-02-07 16:55:45,994 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/fe4cb55689c14489bf281b808521e64d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:55:45,995 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 37ms, sequenceid=45, compaction requested=true 2023-02-07 16:55:45,995 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:45,995 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:45,995 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:45,995 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:45,995 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:45,995 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:45,995 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,007 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,008 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,008 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 16:55:46,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78f6b852[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:46,010 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 16:55:46,010 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,018 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,018 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,018 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:46,028 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,028 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,028 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,028 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,028 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,028 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,039 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,039 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,039 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,039 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:55:46,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 16:55:46,041 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 16:55:46,041 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:46,041 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:46,041 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 16:55:46,041 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:46,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,050 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,050 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,050 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,050 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,051 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,052 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,053 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,053 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,060 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,060 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 16:55:46,060 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:46,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,060 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,061 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,161 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:46,162 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,162 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:46,172 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,173 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,173 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:46,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:46,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 16:55:46,175 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 16:55:46,175 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,175 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 16:55:46,175 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 16:55:46,175 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,175 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,186 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,186 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,186 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,186 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,187 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,187 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:46,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:46,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:46,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:46,188 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:46,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:46,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:46,188 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,188 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:46,188 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. started... 2023-02-07 16:55:46,188 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,188 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:46,196 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,198 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/81fef777a9ff4a05abc6ccceb99b2f52 2023-02-07 16:55:46,207 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,207 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,207 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,207 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,207 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,209 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/ddeb471762134664b50ff54bbef9ecc3 2023-02-07 16:55:46,213 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/81fef777a9ff4a05abc6ccceb99b2f52 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/81fef777a9ff4a05abc6ccceb99b2f52 2023-02-07 16:55:46,216 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/81fef777a9ff4a05abc6ccceb99b2f52, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:55:46,217 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/ddeb471762134664b50ff54bbef9ecc3 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/ddeb471762134664b50ff54bbef9ecc3 2023-02-07 16:55:46,221 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/ddeb471762134664b50ff54bbef9ecc3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:55:46,222 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 34ms, sequenceid=58, compaction requested=true 2023-02-07 16:55:46,222 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:46,222 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:46,222 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,258 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,258 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,259 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 16:55:46,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@840a148[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:46,262 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 16:55:46,262 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,316 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,316 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:46,316 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,317 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:46,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,317 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,317 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,317 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,317 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,317 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:46,317 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:46,333 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,333 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,334 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,334 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,344 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,344 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,344 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,344 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,344 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,344 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:55:46,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 16:55:46,353 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 16:55:46,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:46,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:46,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 16:55:46,354 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:46,365 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,365 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,365 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,365 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,365 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,365 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,366 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,366 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,367 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,368 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,372 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,372 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,373 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,387 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,387 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,387 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,387 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,387 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,387 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 16:55:46,387 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:46,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,386 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,388 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:46,388 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:46,387 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,388 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,388 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,388 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,388 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,389 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,488 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:46,488 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,489 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:46,511 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,511 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:46,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:46,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-07 16:55:46,514 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-07 16:55:46,514 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,514 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-07 16:55:46,514 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-07 16:55:46,515 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,515 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,523 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,523 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,523 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,523 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,524 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,524 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:46,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:46,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:46,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:46,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-07 16:55:46,526 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:46,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-07 16:55:46,526 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,526 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-07 16:55:46,527 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:46,528 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:46,528 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,528 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. started... 2023-02-07 16:55:46,528 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:46,534 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,540 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/305676397e8b426b810b03d4f876f7c6 2023-02-07 16:55:46,544 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,544 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,544 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,544 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,545 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,555 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/f11e3b2f2a1947f8a9fece6ca5e86b62 2023-02-07 16:55:46,560 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/305676397e8b426b810b03d4f876f7c6 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/305676397e8b426b810b03d4f876f7c6 2023-02-07 16:55:46,565 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/305676397e8b426b810b03d4f876f7c6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:55:46,565 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/f11e3b2f2a1947f8a9fece6ca5e86b62 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/f11e3b2f2a1947f8a9fece6ca5e86b62 2023-02-07 16:55:46,569 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/f11e3b2f2a1947f8a9fece6ca5e86b62, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:55:46,570 INFO [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 42ms, sequenceid=71, compaction requested=true 2023-02-07 16:55:46,570 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:46,570 DEBUG [rs(jenkins-hbase12.apache.org,43011,1675788846300)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,570 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:46,570 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:46,570 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,570 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-07 16:55:46,570 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-07 16:55:46,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,576 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,576 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,576 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-07 16:55:46,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c16e848[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:46,579 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-07 16:55:46,579 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,586 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,587 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,587 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:46,586 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:46,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-07 16:55:46,587 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:46,597 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,597 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,597 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,607 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,607 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,607 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,608 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,618 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-07 16:55:46,618 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,618 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-07 16:55:46,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,43011,1675788846300': 2023-02-07 16:55:46,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-07 16:55:46,629 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-07 16:55:46,629 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:46,629 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:46,629 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-07 16:55:46,629 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:46,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,640 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,641 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,642 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:46,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-07 16:55:46,645 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,646 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,647 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,660 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,661 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,661 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,661 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-07 16:55:46,661 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:46,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,661 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-07 16:55:46,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:46,660 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:46,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:46,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:46,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,661 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,662 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:46,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:46,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:46,662 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,662 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:46,662 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:46,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-07 16:55:46,663 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-07 16:55:46,763 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:46,763 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,763 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:46,771 INFO [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,772 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/c1580962b5904ffd892bc7df5edd99bf because midkey is the same as first or last row 2023-02-07 16:55:46,772 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/9208d4928fab48faa418fc64c3326b45 because midkey is the same as first or last row 2023-02-07 16:55:46,772 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegion(2744): Flushing a5e454f361915b4710ba3b438c9f0076 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:55:46,784 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/6723e1d76cd042308487d091d57f60d2 2023-02-07 16:55:46,798 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/c319105108df4b3fa317310cf8cc9b25 2023-02-07 16:55:46,802 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf1/6723e1d76cd042308487d091d57f60d2 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/6723e1d76cd042308487d091d57f60d2 2023-02-07 16:55:46,805 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/6723e1d76cd042308487d091d57f60d2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:55:46,806 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/.tmp/cf2/c319105108df4b3fa317310cf8cc9b25 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/c319105108df4b3fa317310cf8cc9b25 2023-02-07 16:55:46,810 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/c319105108df4b3fa317310cf8cc9b25, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:55:46,810 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a5e454f361915b4710ba3b438c9f0076 in 38ms, sequenceid=82, compaction requested=true 2023-02-07 16:55:46,811 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.HRegion(2445): Flush status journal for a5e454f361915b4710ba3b438c9f0076: 2023-02-07 16:55:46,811 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/c1580962b5904ffd892bc7df5edd99bf because midkey is the same as first or last row 2023-02-07 16:55:46,811 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43011] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/9208d4928fab48faa418fc64c3326b45 because midkey is the same as first or last row 2023-02-07 16:55:46,812 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,812 DEBUG [Listener at localhost.localdomain/34435] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-07 16:55:46,812 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:46,812 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,812 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,814 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,814 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,814 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-07 16:55:46,815 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testInvalidSplitKey 2023-02-07 16:55:46,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:46,818 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,818 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-07 16:55:46,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788946821"}]},"ts":"1675788946821"} 2023-02-07 16:55:46,822 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-07 16:55:46,828 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-07 16:55:46,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, UNASSIGN}] 2023-02-07 16:55:46,830 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, UNASSIGN 2023-02-07 16:55:46,830 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=a5e454f361915b4710ba3b438c9f0076, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:46,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675788946830"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788946830"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788946830"}]},"ts":"1675788946830"} 2023-02-07 16:55:46,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure a5e454f361915b4710ba3b438c9f0076, server=jenkins-hbase12.apache.org,43011,1675788846300}] 2023-02-07 16:55:46,919 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:46,920 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:46,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-07 16:55:46,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:46,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing a5e454f361915b4710ba3b438c9f0076, disabling compactions & flushes 2023-02-07 16:55:46,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. after waiting 0 ms 2023-02-07 16:55:46,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:55:46,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:46,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076. 2023-02-07 16:55:46,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for a5e454f361915b4710ba3b438c9f0076: Waiting for close lock at 1675788946983Running coprocessor pre-close hooks at 1675788946983Disabling compacts and flushes for region at 1675788946983Disabling writes for close at 1675788946984 (+1 ms)Writing region close event to WAL at 1675788946985 (+1 ms)Running coprocessor post-close hooks at 1675788946989 (+4 ms)Closed at 1675788946989 2023-02-07 16:55:46,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:46,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=a5e454f361915b4710ba3b438c9f0076, regionState=CLOSED 2023-02-07 16:55:46,991 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675788946991"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788946991"}]},"ts":"1675788946991"} 2023-02-07 16:55:46,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-07 16:55:46,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure a5e454f361915b4710ba3b438c9f0076, server=jenkins-hbase12.apache.org,43011,1675788846300 in 161 msec 2023-02-07 16:55:46,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-07 16:55:46,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=a5e454f361915b4710ba3b438c9f0076, UNASSIGN in 166 msec 2023-02-07 16:55:46,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788946997"}]},"ts":"1675788946997"} 2023-02-07 16:55:46,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-07 16:55:47,007 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-07 16:55:47,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 193 msec 2023-02-07 16:55:47,123 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:47,123 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:47,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-07 16:55:47,126 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-07 16:55:47,126 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:47,126 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:47,128 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testInvalidSplitKey 2023-02-07 16:55:47,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:47,133 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:47,133 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:47,134 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:47,134 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:47,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-07 16:55:47,140 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:47,143 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/recovered.edits] 2023-02-07 16:55:47,150 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/0a2856deedd443c195c8c6a365d5c29f to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/0a2856deedd443c195c8c6a365d5c29f 2023-02-07 16:55:47,152 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/16ea0099906f49c8bbe2084a124f9a44 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/16ea0099906f49c8bbe2084a124f9a44 2023-02-07 16:55:47,154 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/305676397e8b426b810b03d4f876f7c6 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/305676397e8b426b810b03d4f876f7c6 2023-02-07 16:55:47,155 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/6723e1d76cd042308487d091d57f60d2 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/6723e1d76cd042308487d091d57f60d2 2023-02-07 16:55:47,156 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/81fef777a9ff4a05abc6ccceb99b2f52 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/81fef777a9ff4a05abc6ccceb99b2f52 2023-02-07 16:55:47,158 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/c1580962b5904ffd892bc7df5edd99bf to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/c1580962b5904ffd892bc7df5edd99bf 2023-02-07 16:55:47,159 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/ebaf8afa801743c78d24db641602e7a9 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf1/ebaf8afa801743c78d24db641602e7a9 2023-02-07 16:55:47,162 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/13e4a6b61ea949c085b4b7325d53d556 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/13e4a6b61ea949c085b4b7325d53d556 2023-02-07 16:55:47,163 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/323c5f62e3f24f7eae57557dafb22687 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/323c5f62e3f24f7eae57557dafb22687 2023-02-07 16:55:47,164 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/9208d4928fab48faa418fc64c3326b45 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/9208d4928fab48faa418fc64c3326b45 2023-02-07 16:55:47,165 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/c319105108df4b3fa317310cf8cc9b25 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/c319105108df4b3fa317310cf8cc9b25 2023-02-07 16:55:47,166 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/ddeb471762134664b50ff54bbef9ecc3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/ddeb471762134664b50ff54bbef9ecc3 2023-02-07 16:55:47,167 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/f11e3b2f2a1947f8a9fece6ca5e86b62 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/f11e3b2f2a1947f8a9fece6ca5e86b62 2023-02-07 16:55:47,167 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/fe4cb55689c14489bf281b808521e64d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/cf2/fe4cb55689c14489bf281b808521e64d 2023-02-07 16:55:47,169 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/recovered.edits/85.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076/recovered.edits/85.seqid 2023-02-07 16:55:47,170 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testInvalidSplitKey/a5e454f361915b4710ba3b438c9f0076 2023-02-07 16:55:47,170 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-07 16:55:47,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:47,173 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-07 16:55:47,176 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-07 16:55:47,177 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:47,177 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-07 16:55:47,177 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788947177"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:47,179 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-07 16:55:47,179 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a5e454f361915b4710ba3b438c9f0076, NAME => 'testInvalidSplitKey,,1675788943967.a5e454f361915b4710ba3b438c9f0076.', STARTKEY => '', ENDKEY => ''}] 2023-02-07 16:55:47,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-07 16:55:47,179 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788947179"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:47,180 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-07 16:55:47,187 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:47,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 59 msec 2023-02-07 16:55:47,235 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:47,236 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:47,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-07 16:55:47,238 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-07 16:55:47,256 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 349) - Thread LEAK? -, OpenFileDescriptor=744 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=308 (was 308), ProcessCount=171 (was 171), AvailableMemoryMB=3003 (was 3017) 2023-02-07 16:55:47,274 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=744, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=171, AvailableMemoryMB=3002 2023-02-07 16:55:47,276 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:47,276 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:47,276 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:47,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:55:47,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:55:47,292 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:47,294 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:55:47,294 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:55:47,296 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,297 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 empty. 2023-02-07 16:55:47,298 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,298 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-07 16:55:47,314 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-07 16:55:47,315 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 431592e25e96dcbc7c867f55aa829628, NAME => 'testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:55:47,331 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:47,331 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 431592e25e96dcbc7c867f55aa829628, disabling compactions & flushes 2023-02-07 16:55:47,331 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,331 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,331 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. after waiting 0 ms 2023-02-07 16:55:47,331 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,331 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,331 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 431592e25e96dcbc7c867f55aa829628: Waiting for close lock at 1675788947331Disabling compacts and flushes for region at 1675788947331Disabling writes for close at 1675788947331Writing region close event to WAL at 1675788947331Closed at 1675788947331 2023-02-07 16:55:47,333 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:55:47,333 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788947333"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788947333"}]},"ts":"1675788947333"} 2023-02-07 16:55:47,335 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:55:47,335 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:55:47,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788947335"}]},"ts":"1675788947335"} 2023-02-07 16:55:47,336 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-07 16:55:47,363 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:55:47,365 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:55:47,365 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:55:47,365 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:55:47,365 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:55:47,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, ASSIGN}] 2023-02-07 16:55:47,367 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, ASSIGN 2023-02-07 16:55:47,368 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:55:47,518 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:55:47,520 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=431592e25e96dcbc7c867f55aa829628, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,521 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788947520"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788947520"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788947520"}]},"ts":"1675788947520"} 2023-02-07 16:55:47,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:47,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 431592e25e96dcbc7c867f55aa829628, NAME => 'testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:55:47,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:47,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:47,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,685 INFO [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,686 DEBUG [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1 2023-02-07 16:55:47,687 DEBUG [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1 2023-02-07 16:55:47,687 INFO [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 431592e25e96dcbc7c867f55aa829628 columnFamilyName cf1 2023-02-07 16:55:47,688 INFO [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] regionserver.HStore(310): Store=431592e25e96dcbc7c867f55aa829628/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:47,688 INFO [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,690 DEBUG [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2 2023-02-07 16:55:47,690 DEBUG [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2 2023-02-07 16:55:47,690 INFO [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 431592e25e96dcbc7c867f55aa829628 columnFamilyName cf2 2023-02-07 16:55:47,691 INFO [StoreOpener-431592e25e96dcbc7c867f55aa829628-1] regionserver.HStore(310): Store=431592e25e96dcbc7c867f55aa829628/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:47,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:47,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:47,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:55:47,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 431592e25e96dcbc7c867f55aa829628; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65446042, jitterRate=-0.024777978658676147}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:47,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 431592e25e96dcbc7c867f55aa829628: Running coprocessor pre-open hook at 1675788947683Writing region info on filesystem at 1675788947683Initializing all the Stores at 1675788947684 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788947684Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788947684Cleaning up temporary data from old regions at 1675788947694 (+10 ms)Cleaning up detritus from prior splits at 1675788947695 (+1 ms)Running coprocessor post-open hooks at 1675788947703 (+8 ms)Region opened successfully at 1675788947703 2023-02-07 16:55:47,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628., pid=111, masterSystemTime=1675788947679 2023-02-07 16:55:47,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=431592e25e96dcbc7c867f55aa829628, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788947706"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788947706"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788947706"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788947706"}]},"ts":"1675788947706"} 2023-02-07 16:55:47,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-07 16:55:47,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 in 183 msec 2023-02-07 16:55:47,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-07 16:55:47,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, ASSIGN in 345 msec 2023-02-07 16:55:47,713 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:55:47,713 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788947713"}]},"ts":"1675788947713"} 2023-02-07 16:55:47,714 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-07 16:55:47,724 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:55:47,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 433 msec 2023-02-07 16:55:47,808 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:47,809 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:47,809 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:47,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 16:55:47,811 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:47,811 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:47,811 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 16:55:47,811 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:55:47,812 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,812 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,818 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:47,818 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:47,818 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:47,818 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:47,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:47,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:47,818 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,818 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:47,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:47,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:47,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:47,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:47,820 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:47,821 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:47,821 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:47,821 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:47,820 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:47,820 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. started... 2023-02-07 16:55:47,822 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:55:47,828 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,833 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 2023-02-07 16:55:47,839 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,839 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:47,839 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,839 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,839 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:47,845 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/d3894c6286a243fc92062dada06ea22f 2023-02-07 16:55:47,850 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 2023-02-07 16:55:47,855 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:55:47,855 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/d3894c6286a243fc92062dada06ea22f as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f 2023-02-07 16:55:47,859 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:55:47,859 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 37ms, sequenceid=6, compaction requested=false 2023-02-07 16:55:47,859 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:47,860 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431592e25e96dcbc7c867f55aa829628: 2023-02-07 16:55:47,860 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:47,860 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:47,860 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:47,860 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,860 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:47,860 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:47,870 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,870 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:47,871 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,871 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 16:55:47,872 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40d9a4a7[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:47,872 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,881 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:47,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:47,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:47,881 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:47,882 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:47,882 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:47,891 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:47,891 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:47,891 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:47,902 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,902 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:47,902 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:47,902 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:47,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:47,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:47,912 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:47,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:47,912 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:47,912 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:47,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:47,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:47,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:47,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:47,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:55:47,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:55:47,919 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 16:55:47,920 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:47,920 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:47,920 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 16:55:47,920 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:47,934 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,934 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:47,934 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:47,934 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,934 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,935 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:47,935 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:47,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:47,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:47,935 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:47,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:47,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:47,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:47,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:47,937 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:47,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:47,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:47,942 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,943 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,943 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,955 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:47,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:47,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:47,955 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:47,955 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:47,956 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:47,956 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:47,955 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:47,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:47,956 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:47,956 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:47,956 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:47,956 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:47,957 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:47,957 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:47,957 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,056 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:48,057 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,057 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:48,084 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,084 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:48,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:48,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 16:55:48,087 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 16:55:48,087 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:55:48,088 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,088 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,100 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,100 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,100 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,100 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,101 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,102 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:48,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,105 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:48,106 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,106 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,106 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. started... 2023-02-07 16:55:48,107 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:48,116 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/0ad05cf962164b2eb506069fa3edd09a 2023-02-07 16:55:48,129 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/035df3590a9840ad93642f41394880dd 2023-02-07 16:55:48,135 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/0ad05cf962164b2eb506069fa3edd09a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a 2023-02-07 16:55:48,139 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,139 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,139 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,140 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,139 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,140 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,140 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:55:48,141 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/035df3590a9840ad93642f41394880dd as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd 2023-02-07 16:55:48,147 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:55:48,148 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 41ms, sequenceid=19, compaction requested=false 2023-02-07 16:55:48,148 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431592e25e96dcbc7c867f55aa829628: 2023-02-07 16:55:48,148 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,148 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:48,148 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:48,148 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,148 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,148 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,165 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,165 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,166 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,167 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 16:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@da0b0c6[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:48,173 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:55:48,173 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,184 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,184 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,184 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,185 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,185 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,185 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,185 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,185 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,186 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,186 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:48,185 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:48,186 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,186 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,186 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:48,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,197 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,207 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,208 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,208 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,207 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,208 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,208 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:55:48,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:55:48,219 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 16:55:48,219 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:48,219 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:48,219 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,219 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:48,228 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,228 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,228 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,228 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,228 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,230 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,230 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,232 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,236 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,237 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,238 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,249 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,250 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,250 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,250 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,250 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,249 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,249 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,250 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:48,250 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,251 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,251 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,251 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,251 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,251 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,251 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,351 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:48,351 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,351 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:48,364 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,364 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:48,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:48,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 16:55:48,366 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,366 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,366 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 16:55:48,366 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:55:48,367 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,367 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,375 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,376 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,375 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,375 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,376 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,376 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,377 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:48,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:48,378 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,378 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. started... 2023-02-07 16:55:48,378 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:48,386 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,387 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a 2023-02-07 16:55:48,396 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,396 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,397 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,397 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,397 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,398 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/ec9ff3831e03481b90d99ba065cedb32 2023-02-07 16:55:48,402 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a 2023-02-07 16:55:48,405 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:55:48,406 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/ec9ff3831e03481b90d99ba065cedb32 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 2023-02-07 16:55:48,410 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:55:48,411 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 33ms, sequenceid=32, compaction requested=true 2023-02-07 16:55:48,411 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431592e25e96dcbc7c867f55aa829628: 2023-02-07 16:55:48,411 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,411 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:48,411 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:48,411 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,411 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,411 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,418 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,418 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,418 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,418 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 16:55:48,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@671adb[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:48,420 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:55:48,420 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,428 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,429 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:48,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:48,429 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:48,439 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,439 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,439 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,439 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,477 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,477 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,477 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,477 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,477 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,477 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:55:48,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:55:48,481 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 16:55:48,481 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:48,481 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:48,481 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,481 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,502 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,503 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,503 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,506 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,506 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,507 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,512 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,512 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,512 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,512 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,513 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,513 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:48,512 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,513 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:48,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,513 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,514 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,514 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,514 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,514 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,514 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,514 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,614 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:48,614 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,614 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:48,625 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,625 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,626 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:48,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:48,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 16:55:48,627 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,627 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,627 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 16:55:48,627 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:55:48,627 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,627 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,639 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,639 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,639 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,639 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:48,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,640 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,640 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,641 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,641 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. started... 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,641 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,641 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:48,648 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/d15cff6891b04b79a29910e3e23caa47 2023-02-07 16:55:48,649 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,649 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,649 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,649 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,649 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,650 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,657 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/6ab9e55d4100434f8ab60d98b8ed0987 2023-02-07 16:55:48,661 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/d15cff6891b04b79a29910e3e23caa47 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 2023-02-07 16:55:48,665 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:55:48,665 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/6ab9e55d4100434f8ab60d98b8ed0987 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 2023-02-07 16:55:48,669 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:55:48,670 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 29ms, sequenceid=45, compaction requested=true 2023-02-07 16:55:48,670 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431592e25e96dcbc7c867f55aa829628: 2023-02-07 16:55:48,670 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,670 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:48,670 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:48,670 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,670 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,670 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,681 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,681 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,681 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,681 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 16:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b48f637[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:48,683 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:55:48,683 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,691 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,691 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,691 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,692 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:48,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,692 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:48,692 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:48,702 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,702 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,702 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,712 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,712 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,712 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,713 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,712 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,713 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:55:48,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:55:48,715 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 16:55:48,715 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:48,715 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:48,715 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,715 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,723 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,723 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,724 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,724 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,725 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,726 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,726 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,733 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,733 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,734 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,734 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,734 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:48,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,734 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:48,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,735 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:48,735 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,835 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:48,835 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,835 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:48,844 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:48,844 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:48,845 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:48,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:48,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 16:55:48,846 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:48,846 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,846 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 16:55:48,846 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:55:48,846 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,847 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,860 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,860 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:48,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,860 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,860 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:48,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:48,862 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,862 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. started... 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,862 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:48,862 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,869 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/8b31470e7d4647fbb65da9081a8585ea 2023-02-07 16:55:48,878 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 2023-02-07 16:55:48,882 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/8b31470e7d4647fbb65da9081a8585ea as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea 2023-02-07 16:55:48,885 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:55:48,886 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 2023-02-07 16:55:48,890 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:55:48,890 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 28ms, sequenceid=58, compaction requested=true 2023-02-07 16:55:48,890 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431592e25e96dcbc7c867f55aa829628: 2023-02-07 16:55:48,890 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:48,891 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:48,891 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:48,891 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,891 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:48,891 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:48,899 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,912 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,912 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,913 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,923 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,923 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,923 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,923 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,923 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,923 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 16:55:48,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76398965[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:48,925 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:55:48,925 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,942 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,942 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:48,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,943 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:48,943 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:48,954 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,955 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,955 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,965 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,965 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,965 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,965 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,965 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:48,966 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:48,966 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:48,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:55:48,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:55:48,973 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 16:55:48,973 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:48,973 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:48,973 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 16:55:48,973 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:48,986 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,986 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:48,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:48,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,986 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,986 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,987 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:48,986 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,988 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,988 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:48,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,988 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:48,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:48,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:48,990 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:48,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:48,993 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:48,993 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:48,994 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:48,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:48,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:49,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,008 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,008 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,007 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:49,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,007 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,008 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:49,008 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,009 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,009 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:49,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,009 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,009 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,010 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,010 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,010 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,010 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,022 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:55:49,035 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid a02ccce2-1098-451e-aa52-03b9bc2cf94f) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:55:49,109 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:49,109 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:49,109 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:49,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:49,119 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:49,120 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:49,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:49,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:49,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-07 16:55:49,122 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:49,122 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:49,122 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-07 16:55:49,122 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-07 16:55:49,122 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,122 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,133 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,133 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,133 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,133 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,134 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,134 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:49,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:49,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-07 16:55:49,135 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:49,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-07 16:55:49,135 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:49,135 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:49,135 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:49,135 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-07 16:55:49,136 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:49,136 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. started... 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:49,136 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:49,136 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:49,144 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,144 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/25cd6de419484d1f84d1bd98a05273ec 2023-02-07 16:55:49,154 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,155 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,155 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:49,155 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,155 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:49,158 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/eb0ea00d8b6541ad87414cec6207b483 2023-02-07 16:55:49,163 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/25cd6de419484d1f84d1bd98a05273ec as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec 2023-02-07 16:55:49,169 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:55:49,170 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/eb0ea00d8b6541ad87414cec6207b483 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 2023-02-07 16:55:49,175 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:55:49,176 INFO [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 40ms, sequenceid=71, compaction requested=true 2023-02-07 16:55:49,176 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431592e25e96dcbc7c867f55aa829628: 2023-02-07 16:55:49,176 DEBUG [rs(jenkins-hbase12.apache.org,41937,1675788846347)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-07 16:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-07 16:55:49,186 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,186 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:49,187 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,187 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:49,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:49,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:49,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:49,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:49,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-07 16:55:49,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e1721e4[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:49,192 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-07 16:55:49,192 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:49,198 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:49,198 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:49,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,198 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:49,198 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:49,198 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:49,197 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,198 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:49,198 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-07 16:55:49,199 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-07 16:55:49,199 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:49,207 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:49,207 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:49,207 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,207 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:49,261 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-07 16:55:49,262 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:49,262 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-07 16:55:49,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:49,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:49,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:49,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,41937,1675788846347': 2023-02-07 16:55:49,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-07 16:55:49,276 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-07 16:55:49,276 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:49,276 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:49,276 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-07 16:55:49,276 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:49,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,315 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:49,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,316 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,317 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,317 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:49,317 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-07 16:55:49,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:49,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,393 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,393 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,393 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:49,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,393 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-07 16:55:49,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:49,392 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:49,392 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:49,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:49,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:49,393 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:49,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,394 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,394 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,394 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-07 16:55:49,394 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:49,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:49,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:49,394 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:49,494 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:49,495 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:49,495 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:49,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:49,764 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-07 16:55:49,765 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:49,766 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:49,767 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:49,767 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:49,767 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:49,767 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:49,767 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:49,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:49,768 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:50,016 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,016 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:50,017 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:50,017 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:50,017 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-07 16:55:50,018 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:50,018 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:50,018 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:50,018 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:50,020 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:50,023 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-07 16:55:50,024 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:50,024 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:50,024 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:50,025 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:50,025 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,026 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:50,026 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,026 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,027 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,027 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:50,028 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,028 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,028 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,029 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:50,029 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:50,029 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,029 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,030 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,030 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:50,030 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,030 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,030 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,031 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:50,031 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,031 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,031 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,031 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:50,032 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,033 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:50,033 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,033 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,033 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,033 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-07 16:55:50,034 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:50,034 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:50,034 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788980034 2023-02-07 16:55:50,035 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:50,036 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,038 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:50,038 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:50,038 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:50,038 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789010038 2023-02-07 16:55:50,038 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789070038 2023-02-07 16:55:50,039 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:55:50,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:50,041 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:50,041 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:50,041 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:50,289 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:50,289 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:50,290 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:55:50,292 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-07 16:55:50,292 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:50,292 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:50,293 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:50,293 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:50,293 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,293 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:50,293 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,294 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,294 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,294 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:50,294 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,294 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,295 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,295 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:50,295 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:50,295 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,295 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,295 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,296 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:50,296 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,296 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,296 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,296 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:50,296 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,297 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,297 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,297 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,297 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:50,297 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,298 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,298 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,298 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,298 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,298 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:50,299 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,299 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:50,299 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,299 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,299 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,299 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-07 16:55:50,300 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:50,300 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:50,301 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788980301 2023-02-07 16:55:50,302 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,302 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:50,305 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:50,305 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:50,305 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:50,305 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789010305 2023-02-07 16:55:50,305 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789070305 2023-02-07 16:55:50,305 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:55:50,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:50,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN}] 2023-02-07 16:55:50,308 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:50,308 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:50,308 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:50,556 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,556 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:50,556 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:50,556 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:50,557 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:50,557 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:50,558 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:50,558 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:50,558 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:50,560 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:50,566 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-07 16:55:50,567 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:50,568 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:50,568 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:50,568 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:50,569 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,569 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:50,569 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,569 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,569 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,570 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:50,570 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,570 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,570 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,570 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:50,571 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:50,571 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,571 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,571 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,571 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:50,571 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,572 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,572 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,572 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:50,572 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,572 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,572 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,573 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,574 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:50,574 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,574 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:50,574 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,574 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,574 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,574 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-02-07 16:55:50,575 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:50,575 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:50,576 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788980576 2023-02-07 16:55:50,577 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:50,577 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,579 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:50,579 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:50,579 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:50,579 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789010579 2023-02-07 16:55:50,579 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789070579 2023-02-07 16:55:50,579 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:55:50,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN}] 2023-02-07 16:55:50,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:50,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:50,582 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=431592e25e96dcbc7c867f55aa829628, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:50,583 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788950582"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788950582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788950582"}]},"ts":"1675788950582"} 2023-02-07 16:55:50,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:50,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 431592e25e96dcbc7c867f55aa829628, disabling compactions & flushes 2023-02-07 16:55:50,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:50,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:50,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. after waiting 0 ms 2023-02-07 16:55:50,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:50,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 431592e25e96dcbc7c867f55aa829628 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:55:50,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/c87b340a207547e185ade85b3c803e5d 2023-02-07 16:55:50,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/f46a0e2560044788b77245fe2b10549e 2023-02-07 16:55:50,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf1/c87b340a207547e185ade85b3c803e5d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d 2023-02-07 16:55:50,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:55:50,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/.tmp/cf2/f46a0e2560044788b77245fe2b10549e as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e 2023-02-07 16:55:50,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:55:50,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 39ms, sequenceid=82, compaction requested=true 2023-02-07 16:55:50,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:55:50,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:50,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. 2023-02-07 16:55:50,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 431592e25e96dcbc7c867f55aa829628: Waiting for close lock at 1675788950736Running coprocessor pre-close hooks at 1675788950736Disabling compacts and flushes for region at 1675788950736Disabling writes for close at 1675788950736Obtaining lock to block concurrent updates at 1675788950737 (+1 ms)Preparing flush snapshotting stores in 431592e25e96dcbc7c867f55aa829628 at 1675788950737Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675788950737Flushing stores of testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. at 1675788950737Flushing 431592e25e96dcbc7c867f55aa829628/cf1: creating writer at 1675788950738 (+1 ms)Flushing 431592e25e96dcbc7c867f55aa829628/cf1: appending metadata at 1675788950740 (+2 ms)Flushing 431592e25e96dcbc7c867f55aa829628/cf1: closing flushed file at 1675788950740Flushing 431592e25e96dcbc7c867f55aa829628/cf2: creating writer at 1675788950749 (+9 ms)Flushing 431592e25e96dcbc7c867f55aa829628/cf2: appending metadata at 1675788950750 (+1 ms)Flushing 431592e25e96dcbc7c867f55aa829628/cf2: closing flushed file at 1675788950750Flushing 431592e25e96dcbc7c867f55aa829628/cf1: reopening flushed file at 1675788950768 (+18 ms)Flushing 431592e25e96dcbc7c867f55aa829628/cf2: reopening flushed file at 1675788950772 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 431592e25e96dcbc7c867f55aa829628 in 39ms, sequenceid=82, compaction requested=true at 1675788950776 (+4 ms)Writing region close event to WAL at 1675788950780 (+4 ms)Running coprocessor post-close hooks at 1675788950783 (+3 ms)Closed at 1675788950784 (+1 ms) 2023-02-07 16:55:50,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,786 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED 2023-02-07 16:55:50,787 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788950786"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788950786"}]},"ts":"1675788950786"} 2023-02-07 16:55:50,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-07 16:55:50,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 in 204 msec 2023-02-07 16:55:50,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-07 16:55:50,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN in 209 msec 2023-02-07 16:55:50,792 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:50,792 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:50,792 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:50,830 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:50,830 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:50,831 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:55:50,832 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-07 16:55:50,832 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:50,832 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:50,832 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:50,832 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:50,833 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:50,833 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:50,833 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,833 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:50,833 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,834 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:50,834 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,834 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:50,834 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,834 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:50,835 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,835 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:50,835 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:50,835 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,835 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:50,836 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,836 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:50,836 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,836 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:50,836 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,836 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:50,837 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,837 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:50,837 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,837 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,838 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:50,838 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,838 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:50,839 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,839 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,839 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:50,840 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:50,840 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,840 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:50,840 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:50,841 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:50,841 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,841 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,841 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,841 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:50,841 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:50,841 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-07 16:55:50,842 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:50,843 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:50,843 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788980843 2023-02-07 16:55:50,843 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:50,845 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:50,847 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:50,847 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:50,847 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:50,847 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789010847 2023-02-07 16:55:50,847 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789070847 2023-02-07 16:55:50,847 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:55:50,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:50,859 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=431592e25e96dcbc7c867f55aa829628, threads=14 2023-02-07 16:55:50,859 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,859 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,859 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,860 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,860 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,860 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,860 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,860 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,861 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,861 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,864 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,864 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,865 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,865 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,993 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,993 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,993 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,993 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,998 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,998 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:50,999 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,001 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,009 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,011 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,011 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,013 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,013 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,013 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,013 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 431592e25e96dcbc7c867f55aa829628 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:55:51,041 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:51,041 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:51,041 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:51,097 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,098 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:51,098 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:51,098 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:51,099 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:51,099 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:51,099 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:51,099 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:51,099 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:51,101 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:51,104 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-07 16:55:51,105 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:51,105 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:51,105 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:51,106 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:51,106 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:51,107 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:51,107 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:51,108 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:51,108 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:51,108 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:51,109 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:51,110 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:51,110 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:51,110 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:51,110 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:51,110 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:51,110 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:51,111 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:51,111 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:51,111 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:51,111 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:51,111 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,111 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,112 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:51,112 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,112 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,112 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:51,112 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:51,113 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:51,113 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:51,113 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,113 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:51,113 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:51,113 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:51,114 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,114 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,114 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,114 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:51,114 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:51,114 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-07 16:55:51,115 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:51,115 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:51,115 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788981115 2023-02-07 16:55:51,115 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:51,117 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,119 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:51,119 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:51,120 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:51,120 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789011120 2023-02-07 16:55:51,120 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789071120 2023-02-07 16:55:51,120 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:55:51,127 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=431592e25e96dcbc7c867f55aa829628, threads=14 2023-02-07 16:55:51,127 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,127 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,127 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,127 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,127 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,127 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,128 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,186 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,187 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,196 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,199 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,199 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,200 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,200 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,201 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,204 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,204 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,204 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,205 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,206 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,580 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e for region: 431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:51,581 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 431592e25e96dcbc7c867f55aa829628 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:55:51,622 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:51,628 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:55:51,630 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:55:51,630 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:51,630 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:51,630 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:51,870 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,871 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:51,871 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:51,871 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:51,872 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:51,872 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:51,872 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:51,872 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:51,873 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:51,874 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:51,877 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-07 16:55:51,877 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:51,877 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:51,878 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:51,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:51,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:51,879 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:51,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:51,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:51,880 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:51,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:51,881 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:51,882 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:51,882 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:51,882 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:51,883 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:51,883 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:51,884 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:51,884 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:51,884 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:51,885 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:51,886 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:51,887 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,887 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,887 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:51,887 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:51,887 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-02-07 16:55:51,888 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:51,888 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:51,889 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788981889 2023-02-07 16:55:51,890 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,890 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:51,892 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:51,892 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:51,892 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:51,892 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789011892 2023-02-07 16:55:51,892 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789071892 2023-02-07 16:55:51,892 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:55:51,894 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:51,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:51,896 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:51,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:51,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:52,040 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid 3f3daf44-903e-4489-b9da-57bdf6b56058) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:55:52,142 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,142 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:52,142 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:52,143 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:52,143 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:52,143 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:52,143 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:52,143 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:52,143 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:52,144 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:55:52,144 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-07 16:55:52,144 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:52,144 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,145 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,146 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,147 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:52,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,148 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:52,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:52,149 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:52,150 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:52,150 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,150 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,150 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,150 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,150 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,150 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-07 16:55:52,151 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:52,152 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:52,152 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788982152 2023-02-07 16:55:52,153 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:52,153 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,155 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:52,156 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:52,156 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:52,156 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789012156 2023-02-07 16:55:52,156 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789072156 2023-02-07 16:55:52,156 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:55:52,156 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:52,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788952158"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675788952158"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675788952158"}]},"ts":"1675788952158"} 2023-02-07 16:55:52,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788952158"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788952158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788952158"}]},"ts":"1675788952158"} 2023-02-07 16:55:52,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788952158"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788952158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788952158"}]},"ts":"1675788952158"} 2023-02-07 16:55:52,160 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:52,160 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:52,160 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:52,406 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,407 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:52,407 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:52,407 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:52,408 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:52,408 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:52,408 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:52,408 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:52,409 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:52,410 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:52,414 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-07 16:55:52,414 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:52,414 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:52,415 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:52,416 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:52,416 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,417 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,418 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,418 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:52,418 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:52,418 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,418 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,419 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,420 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,420 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,420 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:52,420 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,421 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,421 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,421 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,421 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,421 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:52,422 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,422 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:52,422 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:52,422 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:52,422 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,422 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,422 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,423 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,423 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,423 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-07 16:55:52,423 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:52,424 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:52,426 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,426 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788982426 2023-02-07 16:55:52,426 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:52,428 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:52,428 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:52,428 INFO [Listener at localhost.localdomain/34435] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347, table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:52,428 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 16:55:52,428 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=0d1a615730f1e4bf719398df929fcdec, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 16:55:52,428 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:52,428 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789012428 2023-02-07 16:55:52,429 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789072429 2023-02-07 16:55:52,429 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-07 16:55:52,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675788952429"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675788952429"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675788952429"}]},"ts":"1675788952429"} 2023-02-07 16:55:52,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788952429"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788952429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788952429"}]},"ts":"1675788952429"} 2023-02-07 16:55:52,429 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788952429"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788952429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788952429"}]},"ts":"1675788952429"} 2023-02-07 16:55:52,431 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:52,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:52,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:52,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:52,679 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,679 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:52,679 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:52,679 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:52,680 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:52,680 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:52,680 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:52,680 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:52,681 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:52,682 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:52,685 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-07 16:55:52,685 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:52,685 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:52,685 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:52,685 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:52,686 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:52,686 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:52,686 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,686 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:52,686 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,686 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,687 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,687 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:52,687 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,687 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,687 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,688 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:52,688 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:52,688 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,688 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,688 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,689 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:52,689 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,689 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,689 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,689 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:52,689 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,690 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,690 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,690 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,690 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:52,690 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,691 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,691 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,691 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,691 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,691 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:52,692 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,692 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:52,692 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:52,692 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:52,692 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,692 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,692 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,692 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,692 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,692 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-07 16:55:52,693 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:52,694 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:52,695 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788982695 2023-02-07 16:55:52,696 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:52,696 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,698 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:52,699 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:52,699 INFO [Listener at localhost.localdomain/34435] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347, table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:52,699 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 16:55:52,699 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=0d1a615730f1e4bf719398df929fcdec, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 16:55:52,699 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:52,699 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789012699 2023-02-07 16:55:52,699 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789072699 2023-02-07 16:55:52,699 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:55:52,699 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:52,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN}] 2023-02-07 16:55:52,701 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:52,701 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:52,701 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:52,949 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,950 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:52,950 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:52,950 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:52,951 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:52,951 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:52,951 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:52,951 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:52,951 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:52,953 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-07 16:55:52,955 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-07 16:55:52,955 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:52,956 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:52,956 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:52,956 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:52,956 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:52,957 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:52,957 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,957 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:52,957 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,957 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:52,958 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,958 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:52,958 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,958 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:52,958 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,958 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:52,959 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:52,959 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,959 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:52,959 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,960 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:52,960 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,960 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:52,960 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,960 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:52,960 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,961 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:52,961 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,961 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,961 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:52,961 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,961 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:52,962 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:52,963 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:52,963 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,963 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,963 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,963 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,963 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:52,963 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-07 16:55:52,964 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:52,964 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:52,965 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788982965 2023-02-07 16:55:52,966 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:52,966 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:52,969 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:52,970 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:52,970 INFO [Listener at localhost.localdomain/34435] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347, table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:52,970 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 16:55:52,970 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=0d1a615730f1e4bf719398df929fcdec, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-07 16:55:52,971 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:52,971 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789012971 2023-02-07 16:55:52,971 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789072971 2023-02-07 16:55:52,971 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-07 16:55:52,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN}] 2023-02-07 16:55:52,971 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:52,973 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN 2023-02-07 16:55:52,974 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:55:52,974 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN 2023-02-07 16:55:52,974 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347; forceNewPlan=false, retain=false 2023-02-07 16:55:53,124 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:55:53,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,126 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788953125"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788953125"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788953125"}]},"ts":"1675788953125"} 2023-02-07 16:55:53,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 57a2a3e2556d7e562b5d0d71a376cf3d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:53,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0d1a615730f1e4bf719398df929fcdec, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,129 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788953129"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788953129"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788953129"}]},"ts":"1675788953129"} 2023-02-07 16:55:53,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 0d1a615730f1e4bf719398df929fcdec, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:53,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 57a2a3e2556d7e562b5d0d71a376cf3d, NAME => 'testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 16:55:53,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:53,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:53,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,284 INFO [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,285 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1 2023-02-07 16:55:53,285 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1 2023-02-07 16:55:53,285 INFO [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57a2a3e2556d7e562b5d0d71a376cf3d columnFamilyName cf1 2023-02-07 16:55:53,303 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84-bottom 2023-02-07 16:55:53,309 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a-bottom 2023-02-07 16:55:53,314 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec-bottom 2023-02-07 16:55:53,318 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea-bottom 2023-02-07 16:55:53,323 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d-bottom 2023-02-07 16:55:53,328 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47-bottom 2023-02-07 16:55:53,334 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a-bottom 2023-02-07 16:55:53,334 INFO [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(310): Store=57a2a3e2556d7e562b5d0d71a376cf3d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:53,334 INFO [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,335 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2 2023-02-07 16:55:53,335 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2 2023-02-07 16:55:53,336 INFO [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57a2a3e2556d7e562b5d0d71a376cf3d columnFamilyName cf2 2023-02-07 16:55:53,355 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd-bottom 2023-02-07 16:55:53,361 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987-bottom 2023-02-07 16:55:53,367 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f-bottom 2023-02-07 16:55:53,372 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483-bottom 2023-02-07 16:55:53,378 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32-bottom 2023-02-07 16:55:53,383 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e-bottom 2023-02-07 16:55:53,388 DEBUG [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395-bottom 2023-02-07 16:55:53,389 INFO [StoreOpener-57a2a3e2556d7e562b5d0d71a376cf3d-1] regionserver.HStore(310): Store=57a2a3e2556d7e562b5d0d71a376cf3d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:53,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:53,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:53,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 57a2a3e2556d7e562b5d0d71a376cf3d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72099487, jitterRate=0.07436607778072357}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:53,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 57a2a3e2556d7e562b5d0d71a376cf3d: Running coprocessor pre-open hook at 1675788953282Writing region info on filesystem at 1675788953282Initializing all the Stores at 1675788953283 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788953283Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788953283Cleaning up temporary data from old regions at 1675788953391 (+108 ms)Cleaning up detritus from prior splits at 1675788953392 (+1 ms)Running coprocessor post-open hooks at 1675788953394 (+2 ms)Region opened successfully at 1675788953394 2023-02-07 16:55:53,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d., pid=117, masterSystemTime=1675788953278 2023-02-07 16:55:53,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:55:53,397 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:55:53,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:55:53,398 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:55:53,398 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,398 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,398 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1912): 57a2a3e2556d7e562b5d0d71a376cf3d/cf2 is initiating minor compaction (all files) 2023-02-07 16:55:53,398 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1912): 57a2a3e2556d7e562b5d0d71a376cf3d/cf1 is initiating minor compaction (all files) 2023-02-07 16:55:53,399 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 57a2a3e2556d7e562b5d0d71a376cf3d/cf2 in testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,399 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 57a2a3e2556d7e562b5d0d71a376cf3d/cf1 in testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,399 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/.tmp, totalSize=34.9 K 2023-02-07 16:55:53,399 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/.tmp, totalSize=34.9 K 2023-02-07 16:55:53,400 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788947797 2023-02-07 16:55:53,400 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788947797 2023-02-07 16:55:53,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:53,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,400 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788948061 2023-02-07 16:55:53,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0d1a615730f1e4bf719398df929fcdec, NAME => 'testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 16:55:53,400 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788948061 2023-02-07 16:55:53,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:53,401 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788948353 2023-02-07 16:55:53,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,401 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788948353 2023-02-07 16:55:53,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:53,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788953401"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788953401"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788953401"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788953401"}]},"ts":"1675788953401"} 2023-02-07 16:55:53,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,401 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788948615 2023-02-07 16:55:53,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,401 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788948615 2023-02-07 16:55:53,402 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788948836 2023-02-07 16:55:53,402 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788948836 2023-02-07 16:55:53,403 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788949111 2023-02-07 16:55:53,403 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788949111 2023-02-07 16:55:53,403 INFO [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-07 16:55:53,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 57a2a3e2556d7e562b5d0d71a376cf3d, server=jenkins-hbase12.apache.org,41937,1675788846347 in 275 msec 2023-02-07 16:55:53,404 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788949498 2023-02-07 16:55:53,404 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788949498 2023-02-07 16:55:53,405 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1 2023-02-07 16:55:53,405 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1 2023-02-07 16:55:53,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN in 433 msec 2023-02-07 16:55:53,406 INFO [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0d1a615730f1e4bf719398df929fcdec columnFamilyName cf1 2023-02-07 16:55:53,419 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] throttle.PressureAwareThroughputController(145): 57a2a3e2556d7e562b5d0d71a376cf3d#cf2#compaction#107 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:55:53,421 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] throttle.PressureAwareThroughputController(145): 57a2a3e2556d7e562b5d0d71a376cf3d#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:55:53,429 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84-top 2023-02-07 16:55:53,435 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a-top 2023-02-07 16:55:53,438 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec-top 2023-02-07 16:55:53,439 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/.tmp/cf2/6db0577e39cd47c1bd7aceab938e3c65 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6db0577e39cd47c1bd7aceab938e3c65 2023-02-07 16:55:53,440 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea-top 2023-02-07 16:55:53,442 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 57a2a3e2556d7e562b5d0d71a376cf3d/cf2 of 57a2a3e2556d7e562b5d0d71a376cf3d into 6db0577e39cd47c1bd7aceab938e3c65(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:55:53,442 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 57a2a3e2556d7e562b5d0d71a376cf3d: 2023-02-07 16:55:53,442 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d., storeName=57a2a3e2556d7e562b5d0d71a376cf3d/cf2, priority=9, startTime=1675788953397; duration=0sec 2023-02-07 16:55:53,442 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:55:53,443 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d-top 2023-02-07 16:55:53,446 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47-top 2023-02-07 16:55:53,449 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a-top 2023-02-07 16:55:53,449 INFO [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(310): Store=0d1a615730f1e4bf719398df929fcdec/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:53,449 INFO [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,450 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2 2023-02-07 16:55:53,450 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2 2023-02-07 16:55:53,450 INFO [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0d1a615730f1e4bf719398df929fcdec columnFamilyName cf2 2023-02-07 16:55:53,463 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd-top 2023-02-07 16:55:53,467 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987-top 2023-02-07 16:55:53,471 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f-top 2023-02-07 16:55:53,475 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483-top 2023-02-07 16:55:53,479 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32-top 2023-02-07 16:55:53,484 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e-top 2023-02-07 16:55:53,489 DEBUG [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395-top 2023-02-07 16:55:53,489 INFO [StoreOpener-0d1a615730f1e4bf719398df929fcdec-1] regionserver.HStore(310): Store=0d1a615730f1e4bf719398df929fcdec/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:53,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:53,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 0d1a615730f1e4bf719398df929fcdec; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71168596, jitterRate=0.06049472093582153}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:53,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 0d1a615730f1e4bf719398df929fcdec: Running coprocessor pre-open hook at 1675788953401Writing region info on filesystem at 1675788953401Initializing all the Stores at 1675788953403 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788953403Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788953403Cleaning up temporary data from old regions at 1675788953492 (+89 ms)Cleaning up detritus from prior splits at 1675788953492Running coprocessor post-open hooks at 1675788953495 (+3 ms)Region opened successfully at 1675788953495 2023-02-07 16:55:53,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec., pid=118, masterSystemTime=1675788953278 2023-02-07 16:55:53,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:55:53,497 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:55:53,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:55:53,497 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,497 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1912): 0d1a615730f1e4bf719398df929fcdec/cf1 is initiating minor compaction (all files) 2023-02-07 16:55:53,497 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0d1a615730f1e4bf719398df929fcdec/cf1 in testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,498 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d-top] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/.tmp, totalSize=34.9 K 2023-02-07 16:55:53,498 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788947797 2023-02-07 16:55:53,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,499 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788948061 2023-02-07 16:55:53,499 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0d1a615730f1e4bf719398df929fcdec, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788953499"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788953499"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788953499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788953499"}]},"ts":"1675788953499"} 2023-02-07 16:55:53,500 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788948353 2023-02-07 16:55:53,500 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788948615 2023-02-07 16:55:53,501 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675788948836 2023-02-07 16:55:53,502 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting 25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675788949111 2023-02-07 16:55:53,502 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] compactions.Compactor(207): Compacting c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788949498 2023-02-07 16:55:53,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-07 16:55:53,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0d1a615730f1e4bf719398df929fcdec, server=jenkins-hbase12.apache.org,41937,1675788846347 in 371 msec 2023-02-07 16:55:53,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-07 16:55:53,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN in 532 msec 2023-02-07 16:55:53,505 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-07 16:55:53,505 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:55:53,505 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-07 16:55:53,510 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] throttle.PressureAwareThroughputController(145): 0d1a615730f1e4bf719398df929fcdec#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:55:53,721 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,721 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-07 16:55:53,721 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:55:53,721 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:55:53,722 ERROR [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-07 16:55:53,723 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-07 16:55:53,723 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-07 16:55:53,723 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-07 16:55:53,723 INFO [Listener at localhost.localdomain/34435] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-07 16:55:53,723 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-07 16:55:53,725 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-07 16:55:53,726 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-07 16:55:53,726 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-07 16:55:53,726 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:53,726 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=ec19d20db6bb6b93bd46e1c22dcf6fe3, daughterA=fb1efbdcd387292fcd3a5366f391724e, daughterB=a4f8504d73cbf3682ed9cc92ec5a306b 2023-02-07 16:55:53,726 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:53,726 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-07 16:55:53,727 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:53,727 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=ec12db53da84ded9dec5333b54c85b5d, daughterA=91836b86732502996543a428936bdda0, daughterB=4b2f65dee3751ab595b2dd0636c28158 2023-02-07 16:55:53,727 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:53,727 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-07 16:55:53,727 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:53,728 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ab96ff60256f44e85e21a1807242ec57, daughterA=495e15228309be9b15c671620a05b627, daughterB=c77b9139a0c759abb3f7951754110a14 2023-02-07 16:55:53,728 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ab96ff60256f44e85e21a1807242ec57, ASSIGN 2023-02-07 16:55:53,728 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:53,728 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-07 16:55:53,728 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=28b4cc61e5f5c6a50e245a62a87d6c2d, daughterA=31438e1bba921e7d9b8bb86997119dbb, daughterB=1a6e4d9b038ccb4c495f43570894b413 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=bc78cd52db0f6344bb0f31fd08c63620, daughterA=79c8a3296cee557a5e6281665f3308bc, daughterB=856cb5044d0fb5e42dada9679874799f 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-07 16:55:53,729 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,40593,1675788846388, table=testRollbackForSplitTableRegionWithReplica, region=38a4d25035f0c2227e54aee84fa910d5 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=7114752e270af1e078cd38e20cc2f45a, daughterA=b8c221e7c21ef101a5769cc077bba519, daughterB=db873e045577e360c85a60cf5a68508e 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:53,730 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-07 16:55:53,731 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:53,731 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 57a2a3e2556d7e562b5d0d71a376cf3d, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0d1a615730f1e4bf719398df929fcdec, server=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec held the lock before restarting, call acquireLock to restore it. 2023-02-07 16:55:53,732 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,732 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,732 INFO [Listener at localhost.localdomain/34435] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 431592e25e96dcbc7c867f55aa829628, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 57a2a3e2556d7e562b5d0d71a376cf3d, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:53,732 DEBUG [Listener at localhost.localdomain/34435] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0d1a615730f1e4bf719398df929fcdec, server=jenkins-hbase12.apache.org,41937,1675788846347 didn't hold the lock before restarting, skip acquiring lock. 2023-02-07 16:55:53,733 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-07 16:55:53,733 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-07 16:55:53,734 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388, table=hbase:meta, region=1588230740 2023-02-07 16:55:53,734 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675788983734 2023-02-07 16:55:53,735 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:53,735 DEBUG [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1519): Joining cluster... 2023-02-07 16:55:53,738 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=602ba3a55336ac4b5e29ccd78323a197, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,43011,1675788846300, regionLocation=jenkins-hbase12.apache.org,43011,1675788846300, openSeqNum=2 2023-02-07 16:55:53,738 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=431592e25e96dcbc7c867f55aa829628, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=2 2023-02-07 16:55:53,738 INFO [Listener at localhost.localdomain/34435] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,41937,1675788846347, table=testRecoveryAndDoubleExecution, region=431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:53,739 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=86 2023-02-07 16:55:53,739 DEBUG [Listener at localhost.localdomain/34435] assignment.RegionStateStore(147): Load hbase:meta entry region=0d1a615730f1e4bf719398df929fcdec, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,41937,1675788846347, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347, openSeqNum=86 2023-02-07 16:55:53,739 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-07 16:55:53,739 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675789013739 2023-02-07 16:55:53,739 INFO [Listener at localhost.localdomain/34435] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675789073739 2023-02-07 16:55:53,739 INFO [Listener at localhost.localdomain/34435] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-07 16:55:53,739 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-07 16:55:53,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=431592e25e96dcbc7c867f55aa829628, daughterA=57a2a3e2556d7e562b5d0d71a376cf3d, daughterB=0d1a615730f1e4bf719398df929fcdec in 3.9740 sec 2023-02-07 16:55:53,847 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/.tmp/cf1/811f2a7a6b68410aa555844d9b672e44 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/811f2a7a6b68410aa555844d9b672e44 2023-02-07 16:55:53,853 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 57a2a3e2556d7e562b5d0d71a376cf3d/cf1 of 57a2a3e2556d7e562b5d0d71a376cf3d into 811f2a7a6b68410aa555844d9b672e44(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:55:53,853 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 57a2a3e2556d7e562b5d0d71a376cf3d: 2023-02-07 16:55:53,853 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d., storeName=57a2a3e2556d7e562b5d0d71a376cf3d/cf1, priority=9, startTime=1675788953395; duration=0sec 2023-02-07 16:55:53,853 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:55:53,853 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:55:53,853 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,853 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1912): 0d1a615730f1e4bf719398df929fcdec/cf2 is initiating minor compaction (all files) 2023-02-07 16:55:53,853 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0d1a615730f1e4bf719398df929fcdec/cf2 in testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:53,854 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e-top] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/.tmp, totalSize=34.9 K 2023-02-07 16:55:53,855 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788947797 2023-02-07 16:55:53,855 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788948061 2023-02-07 16:55:53,856 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788948353 2023-02-07 16:55:53,856 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting 6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788948615 2023-02-07 16:55:53,857 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675788948836 2023-02-07 16:55:53,858 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675788949111 2023-02-07 16:55:53,858 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] compactions.Compactor(207): Compacting f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788949498 2023-02-07 16:55:53,869 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0d1a615730f1e4bf719398df929fcdec#cf2#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:55:53,883 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/.tmp/cf2/ce0066633abd45c697af3604d29f0d9b as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ce0066633abd45c697af3604d29f0d9b 2023-02-07 16:55:53,889 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0d1a615730f1e4bf719398df929fcdec/cf2 of 0d1a615730f1e4bf719398df929fcdec into ce0066633abd45c697af3604d29f0d9b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:55:53,889 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0d1a615730f1e4bf719398df929fcdec: 2023-02-07 16:55:53,889 INFO [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec., storeName=0d1a615730f1e4bf719398df929fcdec/cf2, priority=9, startTime=1675788953497; duration=0sec 2023-02-07 16:55:53,889 DEBUG [RS:1;jenkins-hbase12:41937-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:55:53,927 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/.tmp/cf1/845689680baa4dd885173b04affc4d51 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/845689680baa4dd885173b04affc4d51 2023-02-07 16:55:53,934 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0d1a615730f1e4bf719398df929fcdec/cf1 of 0d1a615730f1e4bf719398df929fcdec into 845689680baa4dd885173b04affc4d51(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:55:53,934 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0d1a615730f1e4bf719398df929fcdec: 2023-02-07 16:55:53,934 INFO [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec., storeName=0d1a615730f1e4bf719398df929fcdec/cf1, priority=9, startTime=1675788953496; duration=0sec 2023-02-07 16:55:53,934 DEBUG [RS:1;jenkins-hbase12:41937-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:55:53,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41937] ipc.CallRunner(144): callId: 866 service: ClientService methodName: Scan size: 154 connection: 136.243.104.168:48258 deadline: 1675789013991, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628. is not online on jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:54,108 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:55:54,118 INFO [Listener at localhost.localdomain/34435] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 16:55:54,119 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:55:54,131 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:54,132 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,132 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,133 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,133 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,134 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-07 16:55:54,134 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRecoveryAndDoubleExecution 2023-02-07 16:55:54,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,136 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,136 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,137 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788954137"}]},"ts":"1675788954137"} 2023-02-07 16:55:54,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-07 16:55:54,138 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-07 16:55:54,184 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-07 16:55:54,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, UNASSIGN}] 2023-02-07 16:55:54,187 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, UNASSIGN 2023-02-07 16:55:54,188 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0d1a615730f1e4bf719398df929fcdec, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:54,188 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788954188"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788954188"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788954188"}]},"ts":"1675788954188"} 2023-02-07 16:55:54,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 0d1a615730f1e4bf719398df929fcdec, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:54,192 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, UNASSIGN 2023-02-07 16:55:54,193 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:54,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788954193"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788954193"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788954193"}]},"ts":"1675788954193"} 2023-02-07 16:55:54,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 57a2a3e2556d7e562b5d0d71a376cf3d, server=jenkins-hbase12.apache.org,41937,1675788846347}] 2023-02-07 16:55:54,237 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,238 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-07 16:55:54,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:54,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 0d1a615730f1e4bf719398df929fcdec, disabling compactions & flushes 2023-02-07 16:55:54,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:54,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:54,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. after waiting 0 ms 2023-02-07 16:55:54,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:54,344 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d-top] to archive 2023-02-07 16:55:54,345 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:55:54,348 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,351 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,352 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,354 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,355 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,356 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,357 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,358 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e-top] to archive 2023-02-07 16:55:54,359 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:55:54,360 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,361 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,362 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,363 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,365 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,366 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,368 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:55:54,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:54,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec. 2023-02-07 16:55:54,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 0d1a615730f1e4bf719398df929fcdec: Waiting for close lock at 1675788954342Running coprocessor pre-close hooks at 1675788954342Disabling compacts and flushes for region at 1675788954342Disabling writes for close at 1675788954343 (+1 ms)Writing region close event to WAL at 1675788954369 (+26 ms)Running coprocessor post-close hooks at 1675788954382 (+13 ms)Closed at 1675788954382 2023-02-07 16:55:54,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:54,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:54,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 57a2a3e2556d7e562b5d0d71a376cf3d, disabling compactions & flushes 2023-02-07 16:55:54,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:54,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:54,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. after waiting 0 ms 2023-02-07 16:55:54,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0d1a615730f1e4bf719398df929fcdec, regionState=CLOSED 2023-02-07 16:55:54,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:54,385 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788954384"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788954384"}]},"ts":"1675788954384"} 2023-02-07 16:55:54,386 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d-bottom] to archive 2023-02-07 16:55:54,386 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:55:54,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-07 16:55:54,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 0d1a615730f1e4bf719398df929fcdec, server=jenkins-hbase12.apache.org,41937,1675788846347 in 196 msec 2023-02-07 16:55:54,388 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0d1a615730f1e4bf719398df929fcdec, UNASSIGN in 201 msec 2023-02-07 16:55:54,389 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,390 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,391 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,392 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,392 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,393 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,395 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e-bottom] to archive 2023-02-07 16:55:54,396 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:55:54,397 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,398 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,398 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,399 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,400 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,400 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,401 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:55:54,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:54,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d. 2023-02-07 16:55:54,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 57a2a3e2556d7e562b5d0d71a376cf3d: Waiting for close lock at 1675788954384Running coprocessor pre-close hooks at 1675788954384Disabling compacts and flushes for region at 1675788954384Disabling writes for close at 1675788954384Writing region close event to WAL at 1675788954402 (+18 ms)Running coprocessor post-close hooks at 1675788954407 (+5 ms)Closed at 1675788954407 2023-02-07 16:55:54,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:54,410 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=57a2a3e2556d7e562b5d0d71a376cf3d, regionState=CLOSED 2023-02-07 16:55:54,410 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675788954410"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788954410"}]},"ts":"1675788954410"} 2023-02-07 16:55:54,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-07 16:55:54,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 57a2a3e2556d7e562b5d0d71a376cf3d, server=jenkins-hbase12.apache.org,41937,1675788846347 in 217 msec 2023-02-07 16:55:54,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-07 16:55:54,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=57a2a3e2556d7e562b5d0d71a376cf3d, UNASSIGN in 226 msec 2023-02-07 16:55:54,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788954413"}]},"ts":"1675788954413"} 2023-02-07 16:55:54,414 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-07 16:55:54,428 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-07 16:55:54,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 294 msec 2023-02-07 16:55:54,439 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,439 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-07 16:55:54,440 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-07 16:55:54,440 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,441 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,442 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRecoveryAndDoubleExecution 2023-02-07 16:55:54,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,445 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,445 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-07 16:55:54,453 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,453 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:54,453 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:54,456 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/recovered.edits] 2023-02-07 16:55:54,457 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/recovered.edits] 2023-02-07 16:55:54,457 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/recovered.edits] 2023-02-07 16:55:54,460 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/01ef2cd45b3c4ab7b0658b23dd520b84 2023-02-07 16:55:54,461 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/.regioninfo to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/.regioninfo 2023-02-07 16:55:54,461 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/.regioninfo to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/.regioninfo 2023-02-07 16:55:54,462 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/0ad05cf962164b2eb506069fa3edd09a 2023-02-07 16:55:54,464 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/25cd6de419484d1f84d1bd98a05273ec 2023-02-07 16:55:54,464 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,464 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/01ef2cd45b3c4ab7b0658b23dd520b84.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,465 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,465 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/8b31470e7d4647fbb65da9081a8585ea 2023-02-07 16:55:54,465 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/0ad05cf962164b2eb506069fa3edd09a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,467 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,467 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/25cd6de419484d1f84d1bd98a05273ec.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,467 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/c87b340a207547e185ade85b3c803e5d 2023-02-07 16:55:54,468 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,469 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/8b31470e7d4647fbb65da9081a8585ea.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,469 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/d15cff6891b04b79a29910e3e23caa47 2023-02-07 16:55:54,470 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,470 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a 2023-02-07 16:55:54,470 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/c87b340a207547e185ade85b3c803e5d.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,471 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,471 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/d15cff6891b04b79a29910e3e23caa47.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,473 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,474 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf1/f0a3ff55701f4634a59c0d1d2ed59e9a.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,474 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/035df3590a9840ad93642f41394880dd 2023-02-07 16:55:54,475 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/6ab9e55d4100434f8ab60d98b8ed0987 2023-02-07 16:55:54,476 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,476 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/035df3590a9840ad93642f41394880dd.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,476 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/d3894c6286a243fc92062dada06ea22f 2023-02-07 16:55:54,477 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,477 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/6ab9e55d4100434f8ab60d98b8ed0987.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,477 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/eb0ea00d8b6541ad87414cec6207b483 2023-02-07 16:55:54,478 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,478 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/d3894c6286a243fc92062dada06ea22f.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,478 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/ec9ff3831e03481b90d99ba065cedb32 2023-02-07 16:55:54,479 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,479 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/eb0ea00d8b6541ad87414cec6207b483.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,479 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/f46a0e2560044788b77245fe2b10549e 2023-02-07 16:55:54,480 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,480 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/ec9ff3831e03481b90d99ba065cedb32.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,480 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/cf2/fd16443f3a8c4c4ea6b2d8591dc11395 2023-02-07 16:55:54,481 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,481 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/f46a0e2560044788b77245fe2b10549e.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,482 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,482 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/0d1a615730f1e4bf719398df929fcdec/cf2/fd16443f3a8c4c4ea6b2d8591dc11395.431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,483 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/recovered.edits/85.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628/recovered.edits/85.seqid 2023-02-07 16:55:54,483 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/431592e25e96dcbc7c867f55aa829628 2023-02-07 16:55:54,484 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/811f2a7a6b68410aa555844d9b672e44 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf1/811f2a7a6b68410aa555844d9b672e44 2023-02-07 16:55:54,484 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/845689680baa4dd885173b04affc4d51 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf1/845689680baa4dd885173b04affc4d51 2023-02-07 16:55:54,486 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6db0577e39cd47c1bd7aceab938e3c65 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/cf2/6db0577e39cd47c1bd7aceab938e3c65 2023-02-07 16:55:54,486 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ce0066633abd45c697af3604d29f0d9b to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/cf2/ce0066633abd45c697af3604d29f0d9b 2023-02-07 16:55:54,488 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d/recovered.edits/90.seqid 2023-02-07 16:55:54,488 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec/recovered.edits/90.seqid 2023-02-07 16:55:54,488 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/57a2a3e2556d7e562b5d0d71a376cf3d 2023-02-07 16:55:54,488 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testRecoveryAndDoubleExecution/0d1a615730f1e4bf719398df929fcdec 2023-02-07 16:55:54,488 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-07 16:55:54,490 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,491 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-07 16:55:54,493 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-07 16:55:54,494 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,494 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-07 16:55:54,494 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788954494"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:54,494 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788954494"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:54,494 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788954494"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:54,496 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:55:54,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 431592e25e96dcbc7c867f55aa829628, NAME => 'testRecoveryAndDoubleExecution,,1675788947291.431592e25e96dcbc7c867f55aa829628.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 57a2a3e2556d7e562b5d0d71a376cf3d, NAME => 'testRecoveryAndDoubleExecution,,1675788949765.57a2a3e2556d7e562b5d0d71a376cf3d.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0d1a615730f1e4bf719398df929fcdec, NAME => 'testRecoveryAndDoubleExecution,41,1675788949765.0d1a615730f1e4bf719398df929fcdec.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 16:55:54,496 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-07 16:55:54,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788954496"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:54,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-07 16:55:54,511 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-07 16:55:54,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 69 msec 2023-02-07 16:55:54,546 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,547 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-07 16:55:54,548 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-07 16:55:54,567 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=375 (was 369) - Thread LEAK? -, OpenFileDescriptor=844 (was 744) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=315 (was 308) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=2928 (was 3002) 2023-02-07 16:55:54,580 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=315, ProcessCount=171, AvailableMemoryMB=2927 2023-02-07 16:55:54,580 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:54,580 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:54,581 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:54,594 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:55:54,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:55:54,596 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:54,599 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:55:54,599 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:55:54,602 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,602 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 empty. 2023-02-07 16:55:54,603 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,603 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-07 16:55:54,618 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-07 16:55:54,619 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0f7a8065107aa1f72d6b1737acaee365, NAME => 'testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:55:54,627 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:54,627 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 0f7a8065107aa1f72d6b1737acaee365, disabling compactions & flushes 2023-02-07 16:55:54,627 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:54,627 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:54,627 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. after waiting 0 ms 2023-02-07 16:55:54,627 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:54,627 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:54,628 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 0f7a8065107aa1f72d6b1737acaee365: Waiting for close lock at 1675788954627Disabling compacts and flushes for region at 1675788954627Disabling writes for close at 1675788954627Writing region close event to WAL at 1675788954627Closed at 1675788954627 2023-02-07 16:55:54,630 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:55:54,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788954631"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788954631"}]},"ts":"1675788954631"} 2023-02-07 16:55:54,632 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:55:54,633 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:55:54,633 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788954633"}]},"ts":"1675788954633"} 2023-02-07 16:55:54,635 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-07 16:55:54,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:55:54,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:55:54,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:55:54,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:55:54,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:55:54,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, ASSIGN}] 2023-02-07 16:55:54,665 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, ASSIGN 2023-02-07 16:55:54,666 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:55:54,817 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:55:54,818 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=0f7a8065107aa1f72d6b1737acaee365, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:54,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788954818"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788954818"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788954818"}]},"ts":"1675788954818"} 2023-02-07 16:55:54,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 0f7a8065107aa1f72d6b1737acaee365, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:54,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:54,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f7a8065107aa1f72d6b1737acaee365, NAME => 'testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:55:54,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:54,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:54,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,990 INFO [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,991 DEBUG [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1 2023-02-07 16:55:54,991 DEBUG [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1 2023-02-07 16:55:54,992 INFO [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f7a8065107aa1f72d6b1737acaee365 columnFamilyName cf1 2023-02-07 16:55:54,992 INFO [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] regionserver.HStore(310): Store=0f7a8065107aa1f72d6b1737acaee365/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:54,992 INFO [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,994 DEBUG [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2 2023-02-07 16:55:54,994 DEBUG [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2 2023-02-07 16:55:54,994 INFO [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f7a8065107aa1f72d6b1737acaee365 columnFamilyName cf2 2023-02-07 16:55:54,995 INFO [StoreOpener-0f7a8065107aa1f72d6b1737acaee365-1] regionserver.HStore(310): Store=0f7a8065107aa1f72d6b1737acaee365/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:54,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:54,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:55,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:55,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:55:55,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 0f7a8065107aa1f72d6b1737acaee365; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67364560, jitterRate=0.0038101673126220703}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:55,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 0f7a8065107aa1f72d6b1737acaee365: Running coprocessor pre-open hook at 1675788954988Writing region info on filesystem at 1675788954988Initializing all the Stores at 1675788954989 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788954989Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788954989Cleaning up temporary data from old regions at 1675788954997 (+8 ms)Cleaning up detritus from prior splits at 1675788954997Running coprocessor post-open hooks at 1675788955002 (+5 ms)Region opened successfully at 1675788955002 2023-02-07 16:55:55,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365., pid=127, masterSystemTime=1675788954974 2023-02-07 16:55:55,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:55,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:55,005 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=0f7a8065107aa1f72d6b1737acaee365, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788955005"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788955005"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788955005"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788955005"}]},"ts":"1675788955005"} 2023-02-07 16:55:55,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-07 16:55:55,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 0f7a8065107aa1f72d6b1737acaee365, server=jenkins-hbase12.apache.org,40593,1675788846388 in 186 msec 2023-02-07 16:55:55,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-07 16:55:55,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, ASSIGN in 348 msec 2023-02-07 16:55:55,013 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:55:55,013 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788955013"}]},"ts":"1675788955013"} 2023-02-07 16:55:55,014 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-07 16:55:55,034 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid 74115c84-26e5-4d15-9a8f-50c13dd5b9f8) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:55:55,146 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:55:55,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 551 msec 2023-02-07 16:55:55,358 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:55,358 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:55,359 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:55,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,360 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:55,360 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,360 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 16:55:55,360 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 16:55:55,361 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,361 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,373 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,373 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,373 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,373 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,374 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,374 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:55,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:55,376 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:55,377 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. started... 2023-02-07 16:55:55,376 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,377 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:55,378 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:55:55,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:55,378 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:55,386 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,391 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd 2023-02-07 16:55:55,396 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,396 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,396 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:55,397 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,397 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:55,405 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/d621980b81d14baab964c3ef21d8b7fa 2023-02-07 16:55:55,411 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd 2023-02-07 16:55:55,417 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:55:55,418 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/d621980b81d14baab964c3ef21d8b7fa as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa 2023-02-07 16:55:55,423 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:55:55,423 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 45ms, sequenceid=6, compaction requested=false 2023-02-07 16:55:55,424 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:55,424 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f7a8065107aa1f72d6b1737acaee365: 2023-02-07 16:55:55,424 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:55,424 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:55,424 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:55,424 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,424 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:55,424 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:55,436 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,436 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:55,437 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,437 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:55,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:55,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 16:55:55,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@461e5d3f[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:55,440 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 16:55:55,440 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:55,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,450 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,450 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:55,449 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:55,450 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:55,460 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:55,460 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:55,461 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:55,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:55,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:55,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:55,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:55:55,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:55:55,469 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 16:55:55,469 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:55,469 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:55,469 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,469 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:55,481 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:55,481 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:55,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,481 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,482 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:55,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:55,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:55,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:55,482 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:55,484 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,485 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,488 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:55,489 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,499 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:55,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:55,582 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:55,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:55,582 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:55,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:55,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,583 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,584 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,584 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:55,584 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:55,584 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,584 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,683 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:55,684 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:55,684 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:55,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:55,701 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:55,702 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:55,703 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:55,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:55,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,705 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:55,705 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,705 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 16:55:55,705 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 16:55:55,705 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,706 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,733 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,733 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,733 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,733 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:55,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:55,734 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:55,734 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,735 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:55,736 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:55,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,736 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,736 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:55,737 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:55,737 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:55,737 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. started... 2023-02-07 16:55:55,738 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:55,744 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,752 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/e9794ab981824b099e84e5b3ee2c97fe 2023-02-07 16:55:55,754 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,754 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:55,754 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:55,755 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,168 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/953d0afa8d11404d84bae99511346595 2023-02-07 16:55:56,179 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/e9794ab981824b099e84e5b3ee2c97fe as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe 2023-02-07 16:55:56,183 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:55:56,184 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/953d0afa8d11404d84bae99511346595 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595 2023-02-07 16:55:56,188 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:55:56,188 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 450ms, sequenceid=19, compaction requested=false 2023-02-07 16:55:56,189 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f7a8065107aa1f72d6b1737acaee365: 2023-02-07 16:55:56,189 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:56,189 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:56,189 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:56,189 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,189 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,189 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,207 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,207 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,207 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,207 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,209 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 16:55:56,209 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a5bf285[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:56,209 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 16:55:56,209 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,217 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,217 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,217 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,218 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:56,218 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:56,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,218 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,219 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,219 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:56,228 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,228 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,228 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,239 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,239 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,239 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,239 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,249 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,249 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,249 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:55:56,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:55:56,261 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 16:55:56,262 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:56,262 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:56,262 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,262 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:56,270 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,270 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,270 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,270 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,271 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,270 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,271 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,271 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,272 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,273 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,275 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,275 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,275 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,285 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,290 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:56,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:56,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,289 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:56,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,290 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:56,291 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,291 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,391 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:56,391 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:56,391 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:56,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:56,403 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:56,403 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:56,404 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:56,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:56,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,405 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:56,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 16:55:56,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 16:55:56,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,426 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,426 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,426 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:56,427 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:56,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,427 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:56,428 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:56,428 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:56,428 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. started... 2023-02-07 16:55:56,428 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:56,440 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/e3b17f5a07624ec18305416a403b5b28 2023-02-07 16:55:56,454 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/2aaaf548bd20465a890ef28e4bdf92ed 2023-02-07 16:55:56,458 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/e3b17f5a07624ec18305416a403b5b28 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28 2023-02-07 16:55:56,461 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:55:56,462 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/2aaaf548bd20465a890ef28e4bdf92ed as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed 2023-02-07 16:55:56,466 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:55:56,466 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 38ms, sequenceid=32, compaction requested=true 2023-02-07 16:55:56,467 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f7a8065107aa1f72d6b1737acaee365: 2023-02-07 16:55:56,467 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:56,467 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:56,467 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:56,467 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,467 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,467 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,468 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,468 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,489 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,489 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,489 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,489 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,489 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,489 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,490 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,490 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 16:55:56,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40932e0a[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:56,491 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 16:55:56,491 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,501 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,502 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:56,502 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:56,512 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,512 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,512 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,512 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,523 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,523 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,523 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,523 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,523 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,523 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:55:56,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:55:56,527 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 16:55:56,527 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:56,528 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:56,528 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,528 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:56,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,534 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,534 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,534 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,534 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,535 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,537 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,537 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,538 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,552 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:56,552 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:56,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,552 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,552 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:56,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,653 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:56,653 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:56,653 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:56,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:56,669 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:56,669 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:56,670 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:56,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:56,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,672 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:56,672 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,672 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 16:55:56,672 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 16:55:56,672 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,672 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,680 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,681 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,681 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,681 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,681 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,681 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:56,683 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:56,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:56,684 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,684 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. started... 2023-02-07 16:55:56,684 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,685 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:56,685 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,685 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,685 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:56,691 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,697 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/8416010e19574ef1b214772031a33574 2023-02-07 16:55:56,702 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,702 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,702 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,702 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,703 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,713 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/31bd5ce84fb6408ba369dba812f3d0d9 2023-02-07 16:55:56,719 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/8416010e19574ef1b214772031a33574 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574 2023-02-07 16:55:56,724 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:55:56,726 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/31bd5ce84fb6408ba369dba812f3d0d9 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9 2023-02-07 16:55:56,732 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:55:56,733 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 48ms, sequenceid=45, compaction requested=true 2023-02-07 16:55:56,734 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f7a8065107aa1f72d6b1737acaee365: 2023-02-07 16:55:56,734 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:56,734 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:56,734 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:56,734 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,734 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:56,734 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:56,744 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,744 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:56,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 16:55:56,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74c2ffd[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:56,749 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 16:55:56,749 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,754 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,755 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,755 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:56,755 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:56,755 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:56,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:56,756 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:56,765 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,765 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,765 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,765 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,775 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,776 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,776 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,786 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:56,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,786 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,787 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:56,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:55:56,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:55:56,800 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 16:55:56,800 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:56,800 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:56,801 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,801 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:56,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,808 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,808 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,809 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:56,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:56,811 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,811 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,812 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,864 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,865 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:56,865 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:56,866 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,865 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:56,866 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:56,867 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:56,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,865 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:56,866 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:56,868 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,868 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:56,869 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:56,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:56,869 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:56,870 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:56,870 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,870 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,967 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:56,967 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:56,967 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:56,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:56,981 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:56,981 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:56,981 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:56,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:56,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,983 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:56,983 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:56,983 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 16:55:56,983 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 16:55:56,983 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:56,983 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,001 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,001 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,002 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,002 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,004 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:57,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:57,004 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:57,005 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:57,005 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. started... 2023-02-07 16:55:57,004 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:57,005 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:57,005 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:57,016 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,027 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,027 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,027 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:57,028 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,028 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:57,037 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid a02ccce2-1098-451e-aa52-03b9bc2cf94f) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:55:57,416 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/6030625d46ed4ff4958830034395e424 2023-02-07 16:55:57,432 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/e46f1f271b244147b516996725e57d31 2023-02-07 16:55:57,438 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/6030625d46ed4ff4958830034395e424 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424 2023-02-07 16:55:57,443 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:55:57,444 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/e46f1f271b244147b516996725e57d31 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31 2023-02-07 16:55:57,449 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:55:57,450 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 445ms, sequenceid=58, compaction requested=true 2023-02-07 16:55:57,450 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f7a8065107aa1f72d6b1737acaee365: 2023-02-07 16:55:57,450 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:57,451 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:57,451 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:57,451 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,451 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:57,451 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:57,469 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,469 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:57,470 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,470 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:57,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:57,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 16:55:57,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@338f7d8e[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:57,474 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 16:55:57,474 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,490 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,490 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,490 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,490 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:57,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:57,490 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,491 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:57,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:57,491 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:57,511 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:57,511 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,511 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:57,532 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:57,532 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,532 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:57,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:57,532 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,533 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:57,532 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,533 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:57,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:57,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:57,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:57,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:55:57,541 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:55:57,541 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 16:55:57,541 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:57,542 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:57,542 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,542 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:57,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,554 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,554 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,553 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:57,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:57,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,555 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:57,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,555 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,557 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:57,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,558 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:57,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,562 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,563 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,563 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,574 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,574 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:57,575 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:57,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,575 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:57,574 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,575 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:57,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,575 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,576 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,676 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:57,676 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:57,676 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:57,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:57,685 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:57,685 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:57,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:55:57,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-07 16:55:57,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,687 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:57,687 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,687 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-07 16:55:57,687 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-07 16:55:57,688 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,688 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,696 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,696 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,696 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,696 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,696 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,696 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:55:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:57,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:57,698 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:57,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-07 16:55:57,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,699 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-07 16:55:57,699 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:55:57,699 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-07 16:55:57,699 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:55:57,699 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:55:57,699 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:57,699 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. started... 2023-02-07 16:55:57,699 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:55:57,709 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/adef2327e50b47829fbf50feba95a52c 2023-02-07 16:55:57,726 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/800790110f244462b41d2e194fcbbb36 2023-02-07 16:55:57,732 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/adef2327e50b47829fbf50feba95a52c as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c 2023-02-07 16:55:57,739 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:55:57,739 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,740 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/800790110f244462b41d2e194fcbbb36 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36 2023-02-07 16:55:57,746 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:55:57,747 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 48ms, sequenceid=71, compaction requested=true 2023-02-07 16:55:57,747 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f7a8065107aa1f72d6b1737acaee365: 2023-02-07 16:55:57,747 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:57,747 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:55:57,747 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:55:57,747 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,747 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-07 16:55:57,748 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-07 16:55:57,759 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,759 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,760 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:57,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,770 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,770 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:57,770 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:57,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:57,771 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,771 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:55:57,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:57,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:57,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:57,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-07 16:55:57,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16e56ea0[Count = 0] remaining members to acquire global barrier 2023-02-07 16:55:57,773 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-07 16:55:57,773 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:55:57,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,785 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:55:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:57,786 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-07 16:55:57,786 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:57,786 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:55:57,786 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-07 16:55:57,786 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:55:57,795 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,795 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:57,795 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:57,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:57,796 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:57,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:57,806 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:57,806 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,806 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-07 16:55:57,806 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,806 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:57,806 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-07 16:55:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:55:57,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-07 16:55:57,815 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-07 16:55:57,815 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:55:57,815 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:55:57,815 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,815 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:55:57,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:55:57,827 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,828 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,830 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:55:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,831 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:55:57,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,835 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,836 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,836 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,848 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:55:57,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,848 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-07 16:55:57,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:55:57,848 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:55:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:55:57,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,849 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-07 16:55:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,849 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,850 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,849 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:55:57,850 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:55:57,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:55:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:55:57,850 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:55:57,850 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,850 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-07 16:55:57,950 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:55:57,950 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:57,950 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:57,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:55:57,967 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,967 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=0f7a8065107aa1f72d6b1737acaee365, daughterA=a502b5d68b3dcb98cdee24a36a76577e, daughterB=09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:57,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=0f7a8065107aa1f72d6b1737acaee365, daughterA=a502b5d68b3dcb98cdee24a36a76577e, daughterB=09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:57,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=0f7a8065107aa1f72d6b1737acaee365, daughterA=a502b5d68b3dcb98cdee24a36a76577e, daughterB=09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:57,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=0f7a8065107aa1f72d6b1737acaee365, daughterA=a502b5d68b3dcb98cdee24a36a76577e, daughterB=09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:57,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, UNASSIGN}] 2023-02-07 16:55:57,972 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, UNASSIGN 2023-02-07 16:55:57,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=0f7a8065107aa1f72d6b1737acaee365, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:57,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788957972"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788957972"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788957972"}]},"ts":"1675788957972"} 2023-02-07 16:55:57,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 0f7a8065107aa1f72d6b1737acaee365, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:58,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 0f7a8065107aa1f72d6b1737acaee365, disabling compactions & flushes 2023-02-07 16:55:58,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:58,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:58,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. after waiting 0 ms 2023-02-07 16:55:58,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:58,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 0f7a8065107aa1f72d6b1737acaee365 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:55:58,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/b147758b3424422abcb476ade837d6d7 2023-02-07 16:55:58,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/0eda3290aafe4c73a5273796f49170ef 2023-02-07 16:55:58,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf1/b147758b3424422abcb476ade837d6d7 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7 2023-02-07 16:55:58,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:55:58,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.tmp/cf2/0eda3290aafe4c73a5273796f49170ef as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef 2023-02-07 16:55:58,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:55:58,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 54ms, sequenceid=82, compaction requested=true 2023-02-07 16:55:58,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:55:58,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:58,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. 2023-02-07 16:55:58,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 0f7a8065107aa1f72d6b1737acaee365: Waiting for close lock at 1675788958126Running coprocessor pre-close hooks at 1675788958126Disabling compacts and flushes for region at 1675788958126Disabling writes for close at 1675788958126Obtaining lock to block concurrent updates at 1675788958127 (+1 ms)Preparing flush snapshotting stores in 0f7a8065107aa1f72d6b1737acaee365 at 1675788958127Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675788958127Flushing stores of testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. at 1675788958128 (+1 ms)Flushing 0f7a8065107aa1f72d6b1737acaee365/cf1: creating writer at 1675788958131 (+3 ms)Flushing 0f7a8065107aa1f72d6b1737acaee365/cf1: appending metadata at 1675788958134 (+3 ms)Flushing 0f7a8065107aa1f72d6b1737acaee365/cf1: closing flushed file at 1675788958134Flushing 0f7a8065107aa1f72d6b1737acaee365/cf2: creating writer at 1675788958152 (+18 ms)Flushing 0f7a8065107aa1f72d6b1737acaee365/cf2: appending metadata at 1675788958155 (+3 ms)Flushing 0f7a8065107aa1f72d6b1737acaee365/cf2: closing flushed file at 1675788958155Flushing 0f7a8065107aa1f72d6b1737acaee365/cf1: reopening flushed file at 1675788958168 (+13 ms)Flushing 0f7a8065107aa1f72d6b1737acaee365/cf2: reopening flushed file at 1675788958175 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0f7a8065107aa1f72d6b1737acaee365 in 54ms, sequenceid=82, compaction requested=true at 1675788958180 (+5 ms)Writing region close event to WAL at 1675788958181 (+1 ms)Running coprocessor post-close hooks at 1675788958184 (+3 ms)Closed at 1675788958185 (+1 ms) 2023-02-07 16:55:58,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=0f7a8065107aa1f72d6b1737acaee365, regionState=CLOSED 2023-02-07 16:55:58,188 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788958187"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788958187"}]},"ts":"1675788958187"} 2023-02-07 16:55:58,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-07 16:55:58,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 0f7a8065107aa1f72d6b1737acaee365, server=jenkins-hbase12.apache.org,40593,1675788846388 in 215 msec 2023-02-07 16:55:58,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-07 16:55:58,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0f7a8065107aa1f72d6b1737acaee365, UNASSIGN in 219 msec 2023-02-07 16:55:58,207 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=0f7a8065107aa1f72d6b1737acaee365, threads=14 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,208 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,209 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,209 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,209 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,209 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,209 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,209 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,215 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe, top=false 2023-02-07 16:55:58,215 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7, top=false 2023-02-07 16:55:58,217 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36, top=false 2023-02-07 16:55:58,219 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31, top=false 2023-02-07 16:55:58,220 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef, top=false 2023-02-07 16:55:58,220 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424, top=false 2023-02-07 16:55:58,221 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595, top=false 2023-02-07 16:55:58,221 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574, top=false 2023-02-07 16:55:58,222 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa, top=false 2023-02-07 16:55:58,222 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9, top=false 2023-02-07 16:55:58,223 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28, top=false 2023-02-07 16:55:58,223 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c, top=false 2023-02-07 16:55:58,224 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed, top=false 2023-02-07 16:55:58,224 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd, top=false 2023-02-07 16:55:58,229 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,229 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,230 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,230 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,230 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,230 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,230 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,231 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,231 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,232 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,232 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,232 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,233 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,234 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,236 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,240 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,240 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,240 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,241 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,241 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,241 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,242 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,242 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,243 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,243 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,245 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/.splits/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595 for child: a502b5d68b3dcb98cdee24a36a76577e, parent: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,246 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,247 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595 for region: 0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:58,247 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 0f7a8065107aa1f72d6b1737acaee365 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-07 16:55:58,277 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:55:58,278 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:55:58,280 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675788958279"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675788958279"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675788958279"}]},"ts":"1675788958279"} 2023-02-07 16:55:58,280 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958279"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788958279"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788958279"}]},"ts":"1675788958279"} 2023-02-07 16:55:58,280 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958279"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788958279"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788958279"}]},"ts":"1675788958279"} 2023-02-07 16:55:58,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, ASSIGN}] 2023-02-07 16:55:58,283 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, ASSIGN 2023-02-07 16:55:58,283 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:55:58,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, ASSIGN 2023-02-07 16:55:58,284 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:55:58,434 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:55:58,437 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=09702fad1de5cded56aeec0c4c3c7913, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958437"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788958437"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788958437"}]},"ts":"1675788958437"} 2023-02-07 16:55:58,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 09702fad1de5cded56aeec0c4c3c7913, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:58,447 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a502b5d68b3dcb98cdee24a36a76577e, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,447 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958447"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788958447"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788958447"}]},"ts":"1675788958447"} 2023-02-07 16:55:58,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure a502b5d68b3dcb98cdee24a36a76577e, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:58,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:58,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 09702fad1de5cded56aeec0c4c3c7913, NAME => 'testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.', STARTKEY => '71', ENDKEY => ''} 2023-02-07 16:55:58,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:58,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:58,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,606 INFO [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,607 DEBUG [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/cf1 2023-02-07 16:55:58,607 DEBUG [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/cf1 2023-02-07 16:55:58,607 INFO [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09702fad1de5cded56aeec0c4c3c7913 columnFamilyName cf1 2023-02-07 16:55:58,608 INFO [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] regionserver.HStore(310): Store=09702fad1de5cded56aeec0c4c3c7913/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:58,608 INFO [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,609 DEBUG [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/cf2 2023-02-07 16:55:58,609 DEBUG [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/cf2 2023-02-07 16:55:58,609 INFO [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09702fad1de5cded56aeec0c4c3c7913 columnFamilyName cf2 2023-02-07 16:55:58,610 INFO [StoreOpener-09702fad1de5cded56aeec0c4c3c7913-1] regionserver.HStore(310): Store=09702fad1de5cded56aeec0c4c3c7913/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:58,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:58,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:58,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 09702fad1de5cded56aeec0c4c3c7913; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63490861, jitterRate=-0.0539124459028244}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:58,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 09702fad1de5cded56aeec0c4c3c7913: Running coprocessor pre-open hook at 1675788958604Writing region info on filesystem at 1675788958604Initializing all the Stores at 1675788958605 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788958605Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788958605Cleaning up temporary data from old regions at 1675788958612 (+7 ms)Cleaning up detritus from prior splits at 1675788958613 (+1 ms)Running coprocessor post-open hooks at 1675788958615 (+2 ms)Region opened successfully at 1675788958615 2023-02-07 16:55:58,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913., pid=133, masterSystemTime=1675788958594 2023-02-07 16:55:58,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:58,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:58,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:58,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a502b5d68b3dcb98cdee24a36a76577e, NAME => 'testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.', STARTKEY => '', ENDKEY => '71'} 2023-02-07 16:55:58,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=09702fad1de5cded56aeec0c4c3c7913, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,619 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958618"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788958618"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788958618"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788958618"}]},"ts":"1675788958618"} 2023-02-07 16:55:58,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:55:58,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:55:58,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,621 INFO [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,622 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1 2023-02-07 16:55:58,622 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1 2023-02-07 16:55:58,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-07 16:55:58,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 09702fad1de5cded56aeec0c4c3c7913, server=jenkins-hbase12.apache.org,40593,1675788846388 in 180 msec 2023-02-07 16:55:58,623 INFO [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a502b5d68b3dcb98cdee24a36a76577e columnFamilyName cf1 2023-02-07 16:55:58,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, ASSIGN in 341 msec 2023-02-07 16:55:58,629 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424 2023-02-07 16:55:58,631 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574 2023-02-07 16:55:58,634 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c 2023-02-07 16:55:58,636 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7 2023-02-07 16:55:58,638 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd 2023-02-07 16:55:58,641 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28 2023-02-07 16:55:58,644 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe 2023-02-07 16:55:58,644 INFO [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(310): Store=a502b5d68b3dcb98cdee24a36a76577e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:58,644 INFO [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,645 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2 2023-02-07 16:55:58,645 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2 2023-02-07 16:55:58,645 INFO [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a502b5d68b3dcb98cdee24a36a76577e columnFamilyName cf2 2023-02-07 16:55:58,649 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef 2023-02-07 16:55:58,651 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed 2023-02-07 16:55:58,654 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9 2023-02-07 16:55:58,657 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36 2023-02-07 16:55:58,660 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595 2023-02-07 16:55:58,663 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa 2023-02-07 16:55:58,667 DEBUG [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31 2023-02-07 16:55:58,667 INFO [StoreOpener-a502b5d68b3dcb98cdee24a36a76577e-1] regionserver.HStore(310): Store=a502b5d68b3dcb98cdee24a36a76577e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:55:58,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:55:58,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:58,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened a502b5d68b3dcb98cdee24a36a76577e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72194017, jitterRate=0.07577468454837799}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:55:58,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for a502b5d68b3dcb98cdee24a36a76577e: Running coprocessor pre-open hook at 1675788958620Writing region info on filesystem at 1675788958620Initializing all the Stores at 1675788958621 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788958621Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788958621Cleaning up temporary data from old regions at 1675788958669 (+48 ms)Cleaning up detritus from prior splits at 1675788958670 (+1 ms)Running coprocessor post-open hooks at 1675788958672 (+2 ms)Region opened successfully at 1675788958672 2023-02-07 16:55:58,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e., pid=134, masterSystemTime=1675788958594 2023-02-07 16:55:58,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:55:58,675 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:55:58,675 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:55:58,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:55:58,675 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 16:55:58,675 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-07 16:55:58,675 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1912): a502b5d68b3dcb98cdee24a36a76577e/cf2 is initiating minor compaction (all files) 2023-02-07 16:55:58,675 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1912): a502b5d68b3dcb98cdee24a36a76577e/cf1 is initiating minor compaction (all files) 2023-02-07 16:55:58,676 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a502b5d68b3dcb98cdee24a36a76577e/cf2 in testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:58,676 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a502b5d68b3dcb98cdee24a36a76577e/cf1 in testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:58,676 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/.tmp, totalSize=34.9 K 2023-02-07 16:55:58,676 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/.tmp, totalSize=34.9 K 2023-02-07 16:55:58,676 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788955351 2023-02-07 16:55:58,676 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788955351 2023-02-07 16:55:58,677 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788955685 2023-02-07 16:55:58,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:58,677 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788955685 2023-02-07 16:55:58,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:58,678 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788956392 2023-02-07 16:55:58,678 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a502b5d68b3dcb98cdee24a36a76577e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,679 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788956392 2023-02-07 16:55:58,679 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958678"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788958678"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788958678"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788958678"}]},"ts":"1675788958678"} 2023-02-07 16:55:58,679 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788956656 2023-02-07 16:55:58,679 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788956656 2023-02-07 16:55:58,680 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788956969 2023-02-07 16:55:58,680 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788957677 2023-02-07 16:55:58,680 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788956969 2023-02-07 16:55:58,681 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788957953 2023-02-07 16:55:58,681 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788957677 2023-02-07 16:55:58,682 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788957953 2023-02-07 16:55:58,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-07 16:55:58,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure a502b5d68b3dcb98cdee24a36a76577e, server=jenkins-hbase12.apache.org,40593,1675788846388 in 233 msec 2023-02-07 16:55:58,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-07 16:55:58,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, ASSIGN in 402 msec 2023-02-07 16:55:58,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=0f7a8065107aa1f72d6b1737acaee365, daughterA=a502b5d68b3dcb98cdee24a36a76577e, daughterB=09702fad1de5cded56aeec0c4c3c7913 in 719 msec 2023-02-07 16:55:58,700 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] throttle.PressureAwareThroughputController(145): a502b5d68b3dcb98cdee24a36a76577e#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:55:58,700 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] throttle.PressureAwareThroughputController(145): a502b5d68b3dcb98cdee24a36a76577e#cf1#compaction#125 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:55:58,711 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/.tmp/cf1/43009f7445ee4352805c1ba31ccac2ff as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/43009f7445ee4352805c1ba31ccac2ff 2023-02-07 16:55:58,715 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a502b5d68b3dcb98cdee24a36a76577e/cf1 of a502b5d68b3dcb98cdee24a36a76577e into 43009f7445ee4352805c1ba31ccac2ff(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:55:58,715 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a502b5d68b3dcb98cdee24a36a76577e: 2023-02-07 16:55:58,715 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e., storeName=a502b5d68b3dcb98cdee24a36a76577e/cf1, priority=9, startTime=1675788958674; duration=0sec 2023-02-07 16:55:58,715 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:55:58,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40593] ipc.CallRunner(144): callId: 997 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:36952 deadline: 1675789018719, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365. is not online on jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,837 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-07 16:55:58,841 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:58,841 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:58,841 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:58,842 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:58,842 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:58,843 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-07 16:55:58,843 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionEmptyDaughter 2023-02-07 16:55:58,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:58,846 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:58,846 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:58,847 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788958847"}]},"ts":"1675788958847"} 2023-02-07 16:55:58,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 16:55:58,848 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-07 16:55:58,858 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-07 16:55:58,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, UNASSIGN}] 2023-02-07 16:55:58,860 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, UNASSIGN 2023-02-07 16:55:58,861 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=09702fad1de5cded56aeec0c4c3c7913, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,861 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958861"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788958861"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788958861"}]},"ts":"1675788958861"} 2023-02-07 16:55:58,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 09702fad1de5cded56aeec0c4c3c7913, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:58,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, UNASSIGN 2023-02-07 16:55:58,865 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a502b5d68b3dcb98cdee24a36a76577e, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:55:58,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788958865"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788958865"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788958865"}]},"ts":"1675788958865"} 2023-02-07 16:55:58,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure a502b5d68b3dcb98cdee24a36a76577e, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:55:58,947 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:58,947 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:58,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 16:55:59,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:59,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing a502b5d68b3dcb98cdee24a36a76577e, disabling compactions & flushes 2023-02-07 16:55:59,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:59,119 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/.tmp/cf2/77b9960f634047c68c5df57498b438cf as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/77b9960f634047c68c5df57498b438cf 2023-02-07 16:55:59,126 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a502b5d68b3dcb98cdee24a36a76577e/cf2 of a502b5d68b3dcb98cdee24a36a76577e into 77b9960f634047c68c5df57498b438cf(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:55:59,126 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a502b5d68b3dcb98cdee24a36a76577e: 2023-02-07 16:55:59,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:59,126 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e., storeName=a502b5d68b3dcb98cdee24a36a76577e/cf2, priority=9, startTime=1675788958675; duration=0sec 2023-02-07 16:55:59,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:59,127 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:55:59,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. after waiting 0 ms 2023-02-07 16:55:59,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:59,127 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7] to archive 2023-02-07 16:55:59,128 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:55:59,131 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-cb033250cc4d4a5ca3a8c8d77e958fbd 2023-02-07 16:55:59,132 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e9794ab981824b099e84e5b3ee2c97fe 2023-02-07 16:55:59,134 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e3b17f5a07624ec18305416a403b5b28 2023-02-07 16:55:59,135 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-8416010e19574ef1b214772031a33574 2023-02-07 16:55:59,137 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-6030625d46ed4ff4958830034395e424 2023-02-07 16:55:59,138 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-adef2327e50b47829fbf50feba95a52c 2023-02-07 16:55:59,140 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-b147758b3424422abcb476ade837d6d7 2023-02-07 16:55:59,142 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef] to archive 2023-02-07 16:55:59,142 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:55:59,144 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-d621980b81d14baab964c3ef21d8b7fa 2023-02-07 16:55:59,145 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-953d0afa8d11404d84bae99511346595 2023-02-07 16:55:59,146 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-2aaaf548bd20465a890ef28e4bdf92ed 2023-02-07 16:55:59,147 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-31bd5ce84fb6408ba369dba812f3d0d9 2023-02-07 16:55:59,148 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-e46f1f271b244147b516996725e57d31 2023-02-07 16:55:59,149 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-800790110f244462b41d2e194fcbbb36 2023-02-07 16:55:59,149 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:59,149 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:59,150 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/testSplitTableRegionEmptyDaughter=0f7a8065107aa1f72d6b1737acaee365-0eda3290aafe4c73a5273796f49170ef 2023-02-07 16:55:59,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 16:55:59,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:55:59,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:59,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e. 2023-02-07 16:55:59,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for a502b5d68b3dcb98cdee24a36a76577e: Waiting for close lock at 1675788959015Running coprocessor pre-close hooks at 1675788959015Disabling compacts and flushes for region at 1675788959015Disabling writes for close at 1675788959127 (+112 ms)Writing region close event to WAL at 1675788959150 (+23 ms)Running coprocessor post-close hooks at 1675788959153 (+3 ms)Closed at 1675788959153 2023-02-07 16:55:59,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:59,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:59,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 09702fad1de5cded56aeec0c4c3c7913, disabling compactions & flushes 2023-02-07 16:55:59,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:59,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:59,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. after waiting 0 ms 2023-02-07 16:55:59,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:59,155 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a502b5d68b3dcb98cdee24a36a76577e, regionState=CLOSED 2023-02-07 16:55:59,155 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788959155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788959155"}]},"ts":"1675788959155"} 2023-02-07 16:55:59,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-07 16:55:59,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-07 16:55:59,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure a502b5d68b3dcb98cdee24a36a76577e, server=jenkins-hbase12.apache.org,40593,1675788846388 in 290 msec 2023-02-07 16:55:59,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:55:59,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913. 2023-02-07 16:55:59,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 09702fad1de5cded56aeec0c4c3c7913: Waiting for close lock at 1675788959154Running coprocessor pre-close hooks at 1675788959154Disabling compacts and flushes for region at 1675788959154Disabling writes for close at 1675788959155 (+1 ms)Writing region close event to WAL at 1675788959155Running coprocessor post-close hooks at 1675788959158 (+3 ms)Closed at 1675788959158 2023-02-07 16:55:59,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a502b5d68b3dcb98cdee24a36a76577e, UNASSIGN in 299 msec 2023-02-07 16:55:59,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:59,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=09702fad1de5cded56aeec0c4c3c7913, regionState=CLOSED 2023-02-07 16:55:59,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675788959160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788959160"}]},"ts":"1675788959160"} 2023-02-07 16:55:59,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-07 16:55:59,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 09702fad1de5cded56aeec0c4c3c7913, server=jenkins-hbase12.apache.org,40593,1675788846388 in 299 msec 2023-02-07 16:55:59,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-07 16:55:59,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09702fad1de5cded56aeec0c4c3c7913, UNASSIGN in 307 msec 2023-02-07 16:55:59,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788959168"}]},"ts":"1675788959168"} 2023-02-07 16:55:59,169 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-07 16:55:59,180 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-07 16:55:59,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 338 msec 2023-02-07 16:55:59,450 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:59,450 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:59,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-07 16:55:59,451 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-07 16:55:59,451 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:59,451 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:59,452 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,455 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:59,455 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:59,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-07 16:55:59,459 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:59,459 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:59,459 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:59,460 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/recovered.edits] 2023-02-07 16:55:59,460 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/recovered.edits] 2023-02-07 16:55:59,460 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/recovered.edits] 2023-02-07 16:55:59,463 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/6030625d46ed4ff4958830034395e424 2023-02-07 16:55:59,463 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/43009f7445ee4352805c1ba31ccac2ff to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf1/43009f7445ee4352805c1ba31ccac2ff 2023-02-07 16:55:59,464 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/8416010e19574ef1b214772031a33574 2023-02-07 16:55:59,465 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/recovered.edits/88.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913/recovered.edits/88.seqid 2023-02-07 16:55:59,465 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/77b9960f634047c68c5df57498b438cf to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/cf2/77b9960f634047c68c5df57498b438cf 2023-02-07 16:55:59,466 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/adef2327e50b47829fbf50feba95a52c 2023-02-07 16:55:59,466 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/09702fad1de5cded56aeec0c4c3c7913 2023-02-07 16:55:59,467 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/b147758b3424422abcb476ade837d6d7 2023-02-07 16:55:59,468 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/cb033250cc4d4a5ca3a8c8d77e958fbd 2023-02-07 16:55:59,468 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e/recovered.edits/90.seqid 2023-02-07 16:55:59,468 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/a502b5d68b3dcb98cdee24a36a76577e 2023-02-07 16:55:59,468 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e3b17f5a07624ec18305416a403b5b28 2023-02-07 16:55:59,469 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf1/e9794ab981824b099e84e5b3ee2c97fe 2023-02-07 16:55:59,471 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/0eda3290aafe4c73a5273796f49170ef 2023-02-07 16:55:59,472 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/2aaaf548bd20465a890ef28e4bdf92ed 2023-02-07 16:55:59,473 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/31bd5ce84fb6408ba369dba812f3d0d9 2023-02-07 16:55:59,474 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/800790110f244462b41d2e194fcbbb36 2023-02-07 16:55:59,475 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/953d0afa8d11404d84bae99511346595 2023-02-07 16:55:59,476 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/d621980b81d14baab964c3ef21d8b7fa 2023-02-07 16:55:59,478 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/cf2/e46f1f271b244147b516996725e57d31 2023-02-07 16:55:59,480 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/recovered.edits/85.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365/recovered.edits/85.seqid 2023-02-07 16:55:59,480 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegionEmptyDaughter/0f7a8065107aa1f72d6b1737acaee365 2023-02-07 16:55:59,480 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-07 16:55:59,482 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,484 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-07 16:55:59,485 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-07 16:55:59,486 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-07 16:55:59,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788959487"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:59,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788959487"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:59,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788959487"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:59,488 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:55:59,488 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0f7a8065107aa1f72d6b1737acaee365, NAME => 'testSplitTableRegionEmptyDaughter,,1675788954595.0f7a8065107aa1f72d6b1737acaee365.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a502b5d68b3dcb98cdee24a36a76577e, NAME => 'testSplitTableRegionEmptyDaughter,,1675788957967.a502b5d68b3dcb98cdee24a36a76577e.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 09702fad1de5cded56aeec0c4c3c7913, NAME => 'testSplitTableRegionEmptyDaughter,71,1675788957967.09702fad1de5cded56aeec0c4c3c7913.', STARTKEY => '71', ENDKEY => ''}] 2023-02-07 16:55:59,488 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-07 16:55:59,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788959488"}]},"ts":"9223372036854775807"} 2023-02-07 16:55:59,490 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-07 16:55:59,539 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-07 16:55:59,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 87 msec 2023-02-07 16:55:59,556 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:59,556 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:59,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-07 16:55:59,557 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-07 16:55:59,568 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374 (was 375), OpenFileDescriptor=844 (was 844), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=290 (was 315), ProcessCount=171 (was 171), AvailableMemoryMB=2919 (was 2927) 2023-02-07 16:55:59,583 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=290, ProcessCount=171, AvailableMemoryMB=2918 2023-02-07 16:55:59,584 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:55:59,585 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:55:59,585 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:55:59,595 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-07 16:55:59,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-07 16:55:59,597 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-07 16:55:59,600 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-07 16:55:59,601 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-07 16:55:59,603 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 2023-02-07 16:55:59,603 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 empty. 2023-02-07 16:55:59,604 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 2023-02-07 16:55:59,604 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-07 16:55:59,620 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-07 16:55:59,621 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 850312242221fdade320e947569b2a49, NAME => 'testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp 2023-02-07 16:56:00,031 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:56:00,032 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 850312242221fdade320e947569b2a49, disabling compactions & flushes 2023-02-07 16:56:00,032 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,032 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,032 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. after waiting 0 ms 2023-02-07 16:56:00,032 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,032 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,032 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 850312242221fdade320e947569b2a49: Waiting for close lock at 1675788960032Disabling compacts and flushes for region at 1675788960032Disabling writes for close at 1675788960032Writing region close event to WAL at 1675788960032Closed at 1675788960032 2023-02-07 16:56:00,037 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid 74115c84-26e5-4d15-9a8f-50c13dd5b9f8) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:56:00,038 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-07 16:56:00,039 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788960039"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788960039"}]},"ts":"1675788960039"} 2023-02-07 16:56:00,040 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-07 16:56:00,041 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-07 16:56:00,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788960041"}]},"ts":"1675788960041"} 2023-02-07 16:56:00,042 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-07 16:56:00,100 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-07 16:56:00,102 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-07 16:56:00,102 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-07 16:56:00,102 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-07 16:56:00,102 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-07 16:56:00,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, ASSIGN}] 2023-02-07 16:56:00,104 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, ASSIGN 2023-02-07 16:56:00,105 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:56:00,255 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-07 16:56:00,257 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=850312242221fdade320e947569b2a49, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788960256"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788960256"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788960256"}]},"ts":"1675788960256"} 2023-02-07 16:56:00,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 850312242221fdade320e947569b2a49, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:56:00,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 850312242221fdade320e947569b2a49, NAME => 'testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.', STARTKEY => '', ENDKEY => ''} 2023-02-07 16:56:00,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:56:00,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 850312242221fdade320e947569b2a49 2023-02-07 16:56:00,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:56:00,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 850312242221fdade320e947569b2a49 2023-02-07 16:56:00,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 850312242221fdade320e947569b2a49 2023-02-07 16:56:00,418 INFO [StoreOpener-850312242221fdade320e947569b2a49-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 850312242221fdade320e947569b2a49 2023-02-07 16:56:00,420 DEBUG [StoreOpener-850312242221fdade320e947569b2a49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1 2023-02-07 16:56:00,420 DEBUG [StoreOpener-850312242221fdade320e947569b2a49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1 2023-02-07 16:56:00,421 INFO [StoreOpener-850312242221fdade320e947569b2a49-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 850312242221fdade320e947569b2a49 columnFamilyName cf1 2023-02-07 16:56:00,421 INFO [StoreOpener-850312242221fdade320e947569b2a49-1] regionserver.HStore(310): Store=850312242221fdade320e947569b2a49/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:56:00,421 INFO [StoreOpener-850312242221fdade320e947569b2a49-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 850312242221fdade320e947569b2a49 2023-02-07 16:56:00,422 DEBUG [StoreOpener-850312242221fdade320e947569b2a49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2 2023-02-07 16:56:00,422 DEBUG [StoreOpener-850312242221fdade320e947569b2a49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2 2023-02-07 16:56:00,422 INFO [StoreOpener-850312242221fdade320e947569b2a49-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 850312242221fdade320e947569b2a49 columnFamilyName cf2 2023-02-07 16:56:00,423 INFO [StoreOpener-850312242221fdade320e947569b2a49-1] regionserver.HStore(310): Store=850312242221fdade320e947569b2a49/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:56:00,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 2023-02-07 16:56:00,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 2023-02-07 16:56:00,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:56:00,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 850312242221fdade320e947569b2a49 2023-02-07 16:56:00,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-07 16:56:00,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 850312242221fdade320e947569b2a49; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61367366, jitterRate=-0.08555498719215393}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:56:00,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 850312242221fdade320e947569b2a49: Running coprocessor pre-open hook at 1675788960417Writing region info on filesystem at 1675788960417Initializing all the Stores at 1675788960418 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788960418Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788960418Cleaning up temporary data from old regions at 1675788960424 (+6 ms)Cleaning up detritus from prior splits at 1675788960424Running coprocessor post-open hooks at 1675788960428 (+4 ms)Region opened successfully at 1675788960428 2023-02-07 16:56:00,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49., pid=143, masterSystemTime=1675788960411 2023-02-07 16:56:00,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=850312242221fdade320e947569b2a49, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788960431"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788960431"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788960431"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788960431"}]},"ts":"1675788960431"} 2023-02-07 16:56:00,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-07 16:56:00,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 850312242221fdade320e947569b2a49, server=jenkins-hbase12.apache.org,40593,1675788846388 in 174 msec 2023-02-07 16:56:00,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-07 16:56:00,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, ASSIGN in 331 msec 2023-02-07 16:56:00,435 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-07 16:56:00,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788960435"}]},"ts":"1675788960435"} 2023-02-07 16:56:00,436 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-07 16:56:00,522 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-07 16:56:00,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 926 msec 2023-02-07 16:56:00,605 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:00,605 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:00,606 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:56:00,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 16:56:00,608 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 16:56:00,608 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,608 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 16:56:00,608 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 16:56:00,608 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,608 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,616 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,616 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,616 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,616 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,619 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,619 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:56:00,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:00,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:00,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:00,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:00,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:00,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:00,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:00,625 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:00,625 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,625 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:00,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:00,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,626 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:56:00,626 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. started... 2023-02-07 16:56:00,626 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:00,627 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:00,627 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:00,629 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-07 16:56:00,637 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,640 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/f74fba1ce04f43329336a31cfc713c52 2023-02-07 16:56:00,648 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,648 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,648 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:00,648 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,648 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:00,653 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/8ca01433cc9e410ab7d23f1e336ee62e 2023-02-07 16:56:00,658 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/f74fba1ce04f43329336a31cfc713c52 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52 2023-02-07 16:56:00,662 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:56:00,663 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/8ca01433cc9e410ab7d23f1e336ee62e as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e 2023-02-07 16:56:00,667 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:56:00,668 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 39ms, sequenceid=6, compaction requested=false 2023-02-07 16:56:00,668 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-07 16:56:00,669 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 850312242221fdade320e947569b2a49: 2023-02-07 16:56:00,669 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,669 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:56:00,669 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:56:00,669 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,669 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:00,669 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:00,679 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,679 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:00,680 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,680 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:00,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 16:56:00,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f5fe108[Count = 0] remaining members to acquire global barrier 2023-02-07 16:56:00,683 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 16:56:00,683 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,720 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:56:00,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:00,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,720 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:00,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:00,721 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:56:00,742 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:00,742 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:00,743 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:00,743 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:00,743 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:00,743 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:00,753 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:00,753 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,753 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:00,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:00,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:00,753 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:00,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:00,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:00,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:00,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:56:00,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 16:56:00,757 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 16:56:00,757 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:56:00,757 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:56:00,757 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 16:56:00,757 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:00,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:00,770 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:00,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:00,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:00,770 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:00,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:00,771 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:00,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:00,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:00,773 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,773 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,773 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:00,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:00,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,785 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 16:56:00,785 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:56:00,785 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 16:56:00,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:00,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:00,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:00,785 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:56:00,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:00,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,786 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,886 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:56:00,886 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:00,886 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:00,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:56:00,896 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:00,896 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:00,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:56:00,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 16:56:00,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 16:56:00,898 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 16:56:00,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 16:56:00,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 16:56:00,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,898 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,928 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,928 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,928 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,928 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:00,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,928 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:00,928 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:56:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:00,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:00,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:00,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:00,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:00,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:00,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:56:00,930 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:56:00,930 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:00,931 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. started... 2023-02-07 16:56:00,931 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:56:00,942 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,944 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/422a6cff692f4b5e96fd0f69d52e4476 2023-02-07 16:56:00,953 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,953 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:00,953 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,953 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:00,953 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:01,362 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/e408284dd6c44d748a9f3c5f64415b11 2023-02-07 16:56:01,367 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/422a6cff692f4b5e96fd0f69d52e4476 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476 2023-02-07 16:56:01,371 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:56:01,372 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/e408284dd6c44d748a9f3c5f64415b11 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11 2023-02-07 16:56:01,377 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11, entries=10, sequenceid=19, filesize=5.0 K 2023-02-07 16:56:01,377 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 446ms, sequenceid=19, compaction requested=false 2023-02-07 16:56:01,378 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 850312242221fdade320e947569b2a49: 2023-02-07 16:56:01,378 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:01,378 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:56:01,378 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:56:01,378 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,378 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:01,378 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:01,395 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,395 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:01,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:01,396 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,396 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:01,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:01,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:01,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:01,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:01,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 16:56:01,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@290a0cd9[Count = 0] remaining members to acquire global barrier 2023-02-07 16:56:01,400 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 16:56:01,400 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,412 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,413 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:01,413 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:01,413 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:56:01,413 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,413 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:56:01,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,413 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:01,413 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:01,413 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:56:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,414 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:01,414 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:01,414 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:56:01,422 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:01,423 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,423 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:01,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:01,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:01,423 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:01,433 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:01,433 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:01,433 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:01,443 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:01,443 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:01,443 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:01,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:01,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:01,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:01,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:56:01,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 16:56:01,448 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 16:56:01,448 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:56:01,448 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:56:01,448 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 16:56:01,448 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:56:01,458 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,458 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,458 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:01,458 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,458 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:01,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:01,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:01,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,459 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:01,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:01,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:01,459 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:01,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:01,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:01,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,460 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:01,461 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:01,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:01,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:01,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,463 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,463 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:01,464 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:01,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:01,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:01,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:01,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 16:56:01,540 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:01,539 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:01,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:01,539 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:01,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:01,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:01,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:01,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:01,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:01,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:01,540 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:01,541 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,541 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,640 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:56:01,641 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:01,641 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:01,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:56:01,652 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:01,652 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:01,653 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:56:01,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 16:56:01,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 16:56:01,654 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 16:56:01,654 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:01,654 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 16:56:01,654 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 16:56:01,655 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,655 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:01,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:01,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:01,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:01,675 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,675 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:01,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:01,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:01,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:01,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:01,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:01,676 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:01,676 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:01,677 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:01,677 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:01,677 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:01,677 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:01,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:01,677 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. started... 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:01,677 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:01,677 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:01,690 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,690 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,690 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,690 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:01,690 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:01,690 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,092 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/797aa2472df54b1490f1e7c87d783ca3 2023-02-07 16:56:02,115 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/55b47ccd73ee4bc49904af82068e7e57 2023-02-07 16:56:02,118 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/797aa2472df54b1490f1e7c87d783ca3 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3 2023-02-07 16:56:02,121 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:56:02,122 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/55b47ccd73ee4bc49904af82068e7e57 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57 2023-02-07 16:56:02,125 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57, entries=10, sequenceid=32, filesize=5.0 K 2023-02-07 16:56:02,125 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 448ms, sequenceid=32, compaction requested=true 2023-02-07 16:56:02,125 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 850312242221fdade320e947569b2a49: 2023-02-07 16:56:02,125 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:02,125 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:56:02,126 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:56:02,126 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,126 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,126 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,279 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,279 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,281 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,281 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 16:56:02,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6121777b[Count = 0] remaining members to acquire global barrier 2023-02-07 16:56:02,288 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 16:56:02,288 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,352 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,352 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,352 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,353 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:02,354 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:02,354 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:56:02,353 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,354 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:56:02,353 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:02,355 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:02,355 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:56:02,354 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:02,355 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:02,355 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:56:02,374 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:02,374 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,374 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:02,385 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:02,385 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,385 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:02,385 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,395 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:02,395 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,395 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:02,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:56:02,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 16:56:02,406 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 16:56:02,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:56:02,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:56:02,406 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 16:56:02,406 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,417 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,416 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,417 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,417 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,419 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,419 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,419 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,426 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,427 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 16:56:02,427 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:56:02,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,427 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 16:56:02,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,428 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:56:02,427 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,428 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,528 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:56:02,528 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:02,528 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:02,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:56:02,545 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:02,545 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:02,546 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:56:02,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 16:56:02,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 16:56:02,548 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 16:56:02,548 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,549 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 16:56:02,549 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 16:56:02,549 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,549 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,559 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,559 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,559 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,559 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,560 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,560 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:56:02,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:02,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,561 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:02,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:02,562 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:02,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:02,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:02,562 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:02,562 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:02,562 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:02,562 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:02,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,563 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,564 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:56:02,564 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:56:02,564 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:02,564 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. started... 2023-02-07 16:56:02,564 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:56:02,574 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,576 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 2023-02-07 16:56:02,584 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,584 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,584 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,585 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,585 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,588 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/451007c8b67a4c9381353241f942109d 2023-02-07 16:56:02,591 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 2023-02-07 16:56:02,595 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:56:02,596 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/451007c8b67a4c9381353241f942109d as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d 2023-02-07 16:56:02,599 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-07 16:56:02,600 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 35ms, sequenceid=45, compaction requested=true 2023-02-07 16:56:02,600 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 850312242221fdade320e947569b2a49: 2023-02-07 16:56:02,600 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:02,600 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:56:02,600 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:56:02,600 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,600 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,600 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,612 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,612 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,612 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,612 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 16:56:02,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c0c9591[Count = 0] remaining members to acquire global barrier 2023-02-07 16:56:02,614 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 16:56:02,614 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,622 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:02,623 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,623 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:56:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:02,623 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:56:02,633 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:02,633 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,633 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,633 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:02,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,710 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:02,710 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,710 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:02,711 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,710 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:02,711 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:56:02,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 16:56:02,714 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 16:56:02,714 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:56:02,714 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:56:02,714 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 16:56:02,714 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:56:02,780 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,780 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,780 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,780 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,781 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,783 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,783 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,784 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,786 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,787 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,801 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,802 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 16:56:02,802 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:56:02,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,802 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:02,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 16:56:02,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:02,803 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,802 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:56:02,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:02,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:02,803 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,803 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,803 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,903 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:56:02,903 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:02,903 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:02,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:56:02,922 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:02,923 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:02,924 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:56:02,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 16:56:02,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 16:56:02,926 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 16:56:02,926 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,926 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 16:56:02,926 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 16:56:02,927 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,927 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,938 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,938 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:02,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:02,939 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,939 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:02,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:02,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:02,939 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:02,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:02,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:02,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:02,939 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:56:02,940 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:56:02,940 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:02,940 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. started... 2023-02-07 16:56:02,940 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:56:02,948 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,949 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,949 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,949 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,949 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,949 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,949 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b 2023-02-07 16:56:02,962 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/6bac32a721024a4094b7df5eef3c59b5 2023-02-07 16:56:02,968 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b 2023-02-07 16:56:02,973 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:56:02,974 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/6bac32a721024a4094b7df5eef3c59b5 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5 2023-02-07 16:56:02,978 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-07 16:56:02,978 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 38ms, sequenceid=58, compaction requested=true 2023-02-07 16:56:02,979 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 850312242221fdade320e947569b2a49: 2023-02-07 16:56:02,979 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:02,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:56:02,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:56:02,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:02,979 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:02,991 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,991 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:02,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:02,991 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:02,991 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:02,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:02,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 16:56:02,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c6b68b9[Count = 0] remaining members to acquire global barrier 2023-02-07 16:56:02,996 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 16:56:02,996 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,007 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:03,007 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,007 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:56:03,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,007 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:03,007 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:03,007 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:56:03,007 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:03,008 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:03,008 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:56:03,007 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:03,008 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:56:03,016 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:03,016 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:03,016 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,016 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,017 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:03,016 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,017 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:03,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:03,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:03,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:03,027 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:03,027 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,027 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:03,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:03,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:03,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:56:03,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 16:56:03,038 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 16:56:03,038 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:56:03,038 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:56:03,038 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 16:56:03,038 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:56:03,048 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,048 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:03,048 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,048 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:03,049 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,048 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,049 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,050 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:03,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,053 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:03,055 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,055 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,055 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 16:56:03,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,069 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:56:03,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,069 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,070 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,169 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:56:03,169 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:03,169 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:03,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:56:03,180 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:03,180 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:03,180 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-07 16:56:03,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-07 16:56:03,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-07 16:56:03,182 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-07 16:56:03,182 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:03,182 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-07 16:56:03,182 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-07 16:56:03,183 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,183 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,195 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,195 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,195 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,195 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,195 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,195 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-07 16:56:03,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:03,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:03,196 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:03,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:03,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-07 16:56:03,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:03,196 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:03,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:03,197 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-07 16:56:03,197 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. started... 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,43011,1675788846300' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:03,197 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,41937,1675788846347' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:03,197 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-07 16:56:03,207 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/b3a394b523f04e55864aaeb1700863a3 2023-02-07 16:56:03,216 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/f51f933030604c68bfcb465d165ee794 2023-02-07 16:56:03,220 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/b3a394b523f04e55864aaeb1700863a3 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3 2023-02-07 16:56:03,223 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:56:03,224 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/f51f933030604c68bfcb465d165ee794 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794 2023-02-07 16:56:03,227 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794, entries=10, sequenceid=71, filesize=5.0 K 2023-02-07 16:56:03,227 INFO [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 30ms, sequenceid=71, compaction requested=true 2023-02-07 16:56:03,227 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 850312242221fdade320e947569b2a49: 2023-02-07 16:56:03,227 DEBUG [rs(jenkins-hbase12.apache.org,40593,1675788846388)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:03,227 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-07 16:56:03,227 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-07 16:56:03,227 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,227 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-07 16:56:03,228 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-07 16:56:03,230 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,247 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,248 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,248 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:03,258 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,258 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,258 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,258 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:03,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:03,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:03,259 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,259 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-07 16:56:03,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:03,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:03,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:03,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,40593,1675788846388' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-07 16:56:03,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15fecd29[Count = 0] remaining members to acquire global barrier 2023-02-07 16:56:03,261 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-07 16:56:03,261 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:03,269 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,269 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-07 16:56:03,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,40593,1675788846388' in zk 2023-02-07 16:56:03,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,41937,1675788846347' in zk 2023-02-07 16:56:03,269 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,43011,1675788846300' in zk 2023-02-07 16:56:03,279 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,279 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:03,279 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:03,279 DEBUG [member: 'jenkins-hbase12.apache.org,40593,1675788846388' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:03,290 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:03,290 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,290 DEBUG [member: 'jenkins-hbase12.apache.org,41937,1675788846347' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:03,300 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-07 16:56:03,300 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,300 DEBUG [member: 'jenkins-hbase12.apache.org,43011,1675788846300' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-07 16:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,40593,1675788846388': 2023-02-07 16:56:03,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,40593,1675788846388' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-07 16:56:03,303 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-07 16:56:03,303 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-07 16:56:03,303 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-07 16:56:03,303 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-07 16:56:03,303 INFO [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-07 16:56:03,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,311 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-07 16:56:03,311 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,312 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,312 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-07 16:56:03,313 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-07 16:56:03,313 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,313 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,321 DEBUG [(jenkins-hbase12.apache.org,39383,1675788844945)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-07 16:56:03,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-07 16:56:03,321 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-07 16:56:03,322 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-07 16:56:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,321 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-07 16:56:03,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-07 16:56:03,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-07 16:56:03,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,322 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,323 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:03,323 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:03,323 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-07 16:56:03,323 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-07 16:56:03,422 DEBUG [Listener at localhost.localdomain/34435] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-07 16:56:03,423 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:03,423 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:03,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-07 16:56:03,430 DEBUG [Listener at localhost.localdomain/34435] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,430 DEBUG [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=850312242221fdade320e947569b2a49, daughterA=88886102836bfd09baf6d56ee561be4c, daughterB=46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:03,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=850312242221fdade320e947569b2a49, daughterA=88886102836bfd09baf6d56ee561be4c, daughterB=46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:03,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=850312242221fdade320e947569b2a49, daughterA=88886102836bfd09baf6d56ee561be4c, daughterB=46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:03,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=850312242221fdade320e947569b2a49, daughterA=88886102836bfd09baf6d56ee561be4c, daughterB=46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:03,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, UNASSIGN}] 2023-02-07 16:56:03,433 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, UNASSIGN 2023-02-07 16:56:03,433 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=850312242221fdade320e947569b2a49, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:03,433 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788963433"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788963433"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788963433"}]},"ts":"1675788963433"} 2023-02-07 16:56:03,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 850312242221fdade320e947569b2a49, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:56:03,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 850312242221fdade320e947569b2a49, disabling compactions & flushes 2023-02-07 16:56:03,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:03,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:03,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. after waiting 0 ms 2023-02-07 16:56:03,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:03,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 850312242221fdade320e947569b2a49 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-07 16:56:03,594 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/3d3052199798497a8fd6841585fd9e71 2023-02-07 16:56:03,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b 2023-02-07 16:56:03,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf1/3d3052199798497a8fd6841585fd9e71 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71 2023-02-07 16:56:03,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:56:03,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/.tmp/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b 2023-02-07 16:56:03,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-07 16:56:03,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 26ms, sequenceid=82, compaction requested=true 2023-02-07 16:56:03,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-07 16:56:03,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:56:03,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. 2023-02-07 16:56:03,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 850312242221fdade320e947569b2a49: Waiting for close lock at 1675788963586Running coprocessor pre-close hooks at 1675788963586Disabling compacts and flushes for region at 1675788963586Disabling writes for close at 1675788963586Obtaining lock to block concurrent updates at 1675788963586Preparing flush snapshotting stores in 850312242221fdade320e947569b2a49 at 1675788963586Finished memstore snapshotting testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675788963586Flushing stores of testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. at 1675788963586Flushing 850312242221fdade320e947569b2a49/cf1: creating writer at 1675788963587 (+1 ms)Flushing 850312242221fdade320e947569b2a49/cf1: appending metadata at 1675788963589 (+2 ms)Flushing 850312242221fdade320e947569b2a49/cf1: closing flushed file at 1675788963589Flushing 850312242221fdade320e947569b2a49/cf2: creating writer at 1675788963596 (+7 ms)Flushing 850312242221fdade320e947569b2a49/cf2: appending metadata at 1675788963598 (+2 ms)Flushing 850312242221fdade320e947569b2a49/cf2: closing flushed file at 1675788963598Flushing 850312242221fdade320e947569b2a49/cf1: reopening flushed file at 1675788963606 (+8 ms)Flushing 850312242221fdade320e947569b2a49/cf2: reopening flushed file at 1675788963609 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 850312242221fdade320e947569b2a49 in 26ms, sequenceid=82, compaction requested=true at 1675788963612 (+3 ms)Writing region close event to WAL at 1675788963614 (+2 ms)Running coprocessor post-close hooks at 1675788963616 (+2 ms)Closed at 1675788963616 2023-02-07 16:56:03,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=850312242221fdade320e947569b2a49, regionState=CLOSED 2023-02-07 16:56:03,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788963618"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788963618"}]},"ts":"1675788963618"} 2023-02-07 16:56:03,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-07 16:56:03,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 850312242221fdade320e947569b2a49, server=jenkins-hbase12.apache.org,40593,1675788846388 in 185 msec 2023-02-07 16:56:03,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-07 16:56:03,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=850312242221fdade320e947569b2a49, UNASSIGN in 188 msec 2023-02-07 16:56:03,628 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=850312242221fdade320e947569b2a49, threads=14 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,629 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,630 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,630 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,631 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,632 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,686 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-07 16:56:03,781 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,824 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,829 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,857 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,861 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,863 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,863 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,865 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,866 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,866 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,867 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,867 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:03,868 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52 for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:04,181 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b for region: 850312242221fdade320e947569b2a49 2023-02-07 16:56:04,181 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 850312242221fdade320e947569b2a49 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-07 16:56:04,233 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:56:04,234 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-07 16:56:04,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675788964235"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675788964235"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675788964235"}]},"ts":"1675788964235"} 2023-02-07 16:56:04,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788964235"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788964235"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788964235"}]},"ts":"1675788964235"} 2023-02-07 16:56:04,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788964235"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788964235"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788964235"}]},"ts":"1675788964235"} 2023-02-07 16:56:04,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, ASSIGN}] 2023-02-07 16:56:04,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, ASSIGN 2023-02-07 16:56:04,241 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:56:04,241 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, ASSIGN 2023-02-07 16:56:04,241 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,40593,1675788846388; forceNewPlan=false, retain=false 2023-02-07 16:56:04,391 INFO [jenkins-hbase12:39383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-07 16:56:04,394 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=46599e7bdfb807fec0bdc3aed3500139, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:04,394 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788964394"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788964394"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788964394"}]},"ts":"1675788964394"} 2023-02-07 16:56:04,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 46599e7bdfb807fec0bdc3aed3500139, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:56:04,402 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=88886102836bfd09baf6d56ee561be4c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:04,402 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788964402"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788964402"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788964402"}]},"ts":"1675788964402"} 2023-02-07 16:56:04,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure 88886102836bfd09baf6d56ee561be4c, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:56:04,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 46599e7bdfb807fec0bdc3aed3500139, NAME => 'testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.', STARTKEY => '41', ENDKEY => ''} 2023-02-07 16:56:04,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:56:04,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:56:04,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,560 INFO [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,562 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1 2023-02-07 16:56:04,562 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1 2023-02-07 16:56:04,563 INFO [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46599e7bdfb807fec0bdc3aed3500139 columnFamilyName cf1 2023-02-07 16:56:04,578 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71-top 2023-02-07 16:56:04,583 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476-top 2023-02-07 16:56:04,587 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3-top 2023-02-07 16:56:04,591 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b-top 2023-02-07 16:56:04,595 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3-top 2023-02-07 16:56:04,598 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178-top 2023-02-07 16:56:04,602 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52-top 2023-02-07 16:56:04,602 INFO [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(310): Store=46599e7bdfb807fec0bdc3aed3500139/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:56:04,602 INFO [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,603 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2 2023-02-07 16:56:04,603 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2 2023-02-07 16:56:04,604 INFO [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46599e7bdfb807fec0bdc3aed3500139 columnFamilyName cf2 2023-02-07 16:56:04,619 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d-top 2023-02-07 16:56:04,624 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57-top 2023-02-07 16:56:04,630 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5-top 2023-02-07 16:56:04,635 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e-top 2023-02-07 16:56:04,639 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11-top 2023-02-07 16:56:04,645 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b-top 2023-02-07 16:56:04,649 DEBUG [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794-top 2023-02-07 16:56:04,649 INFO [StoreOpener-46599e7bdfb807fec0bdc3aed3500139-1] regionserver.HStore(310): Store=46599e7bdfb807fec0bdc3aed3500139/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:56:04,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:56:04,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:04,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 46599e7bdfb807fec0bdc3aed3500139; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71649653, jitterRate=0.0676630288362503}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:56:04,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 46599e7bdfb807fec0bdc3aed3500139: Running coprocessor pre-open hook at 1675788964558Writing region info on filesystem at 1675788964558Initializing all the Stores at 1675788964560 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788964560Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788964560Cleaning up temporary data from old regions at 1675788964652 (+92 ms)Cleaning up detritus from prior splits at 1675788964652Running coprocessor post-open hooks at 1675788964654 (+2 ms)Region opened successfully at 1675788964654 2023-02-07 16:56:04,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139., pid=149, masterSystemTime=1675788964550 2023-02-07 16:56:04,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:56:04,656 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:56:04,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:56:04,656 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,656 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:56:04,656 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1912): 46599e7bdfb807fec0bdc3aed3500139/cf1 is initiating minor compaction (all files) 2023-02-07 16:56:04,656 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,656 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1912): 46599e7bdfb807fec0bdc3aed3500139/cf2 is initiating minor compaction (all files) 2023-02-07 16:56:04,656 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 46599e7bdfb807fec0bdc3aed3500139/cf1 in testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,656 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 46599e7bdfb807fec0bdc3aed3500139/cf2 in testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,657 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b-top] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/.tmp, totalSize=34.9 K 2023-02-07 16:56:04,657 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71-top] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/.tmp, totalSize=34.9 K 2023-02-07 16:56:04,657 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788960600 2023-02-07 16:56:04,657 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675788960600 2023-02-07 16:56:04,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:04,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 88886102836bfd09baf6d56ee561be4c, NAME => 'testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.', STARTKEY => '', ENDKEY => '41'} 2023-02-07 16:56:04,658 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=46599e7bdfb807fec0bdc3aed3500139, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:04,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-07 16:56:04,658 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788964658"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788964658"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788964658"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788964658"}]},"ts":"1675788964658"} 2023-02-07 16:56:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-07 16:56:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-07 16:56:04,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 46599e7bdfb807fec0bdc3aed3500139, server=jenkins-hbase12.apache.org,40593,1675788846388 in 262 msec 2023-02-07 16:56:04,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, ASSIGN in 422 msec 2023-02-07 16:56:04,661 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788960888 2023-02-07 16:56:04,661 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675788960888 2023-02-07 16:56:04,661 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788961642 2023-02-07 16:56:04,661 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675788961642 2023-02-07 16:56:04,661 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788962530 2023-02-07 16:56:04,661 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675788962530 2023-02-07 16:56:04,662 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675788962905 2023-02-07 16:56:04,662 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675788962905 2023-02-07 16:56:04,662 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675788963171 2023-02-07 16:56:04,662 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675788963171 2023-02-07 16:56:04,662 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788963424 2023-02-07 16:56:04,662 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675788963424 2023-02-07 16:56:04,665 INFO [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,666 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1 2023-02-07 16:56:04,666 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1 2023-02-07 16:56:04,667 INFO [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88886102836bfd09baf6d56ee561be4c columnFamilyName cf1 2023-02-07 16:56:04,670 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] throttle.PressureAwareThroughputController(145): 46599e7bdfb807fec0bdc3aed3500139#cf2#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:56:04,675 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46599e7bdfb807fec0bdc3aed3500139#cf1#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:56:04,678 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71-bottom 2023-02-07 16:56:04,680 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476-bottom 2023-02-07 16:56:04,683 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3-bottom 2023-02-07 16:56:04,690 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/.tmp/cf2/2c3c03772e704ae6b8966e848ae17711 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/2c3c03772e704ae6b8966e848ae17711 2023-02-07 16:56:04,691 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b-bottom 2023-02-07 16:56:04,694 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3-bottom 2023-02-07 16:56:04,695 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/.tmp/cf1/8fee70aa00a045aca0766f9d6d5511cf as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/8fee70aa00a045aca0766f9d6d5511cf 2023-02-07 16:56:04,697 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46599e7bdfb807fec0bdc3aed3500139/cf2 of 46599e7bdfb807fec0bdc3aed3500139 into 2c3c03772e704ae6b8966e848ae17711(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:56:04,697 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 46599e7bdfb807fec0bdc3aed3500139: 2023-02-07 16:56:04,697 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139., storeName=46599e7bdfb807fec0bdc3aed3500139/cf2, priority=9, startTime=1675788964656; duration=0sec 2023-02-07 16:56:04,697 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:56:04,698 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178-bottom 2023-02-07 16:56:04,699 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46599e7bdfb807fec0bdc3aed3500139/cf1 of 46599e7bdfb807fec0bdc3aed3500139 into 8fee70aa00a045aca0766f9d6d5511cf(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:56:04,700 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 46599e7bdfb807fec0bdc3aed3500139: 2023-02-07 16:56:04,700 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139., storeName=46599e7bdfb807fec0bdc3aed3500139/cf1, priority=9, startTime=1675788964655; duration=0sec 2023-02-07 16:56:04,700 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:56:04,701 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52-bottom 2023-02-07 16:56:04,701 INFO [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(310): Store=88886102836bfd09baf6d56ee561be4c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:56:04,701 INFO [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,702 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2 2023-02-07 16:56:04,702 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2 2023-02-07 16:56:04,702 INFO [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88886102836bfd09baf6d56ee561be4c columnFamilyName cf2 2023-02-07 16:56:04,716 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d-bottom 2023-02-07 16:56:04,719 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57-bottom 2023-02-07 16:56:04,721 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5-bottom 2023-02-07 16:56:04,724 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e-bottom 2023-02-07 16:56:04,728 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11-bottom 2023-02-07 16:56:04,731 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b-bottom 2023-02-07 16:56:04,734 DEBUG [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794-bottom 2023-02-07 16:56:04,734 INFO [StoreOpener-88886102836bfd09baf6d56ee561be4c-1] regionserver.HStore(310): Store=88886102836bfd09baf6d56ee561be4c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-07 16:56:04,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-07 16:56:04,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:04,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 88886102836bfd09baf6d56ee561be4c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65770194, jitterRate=-0.019947737455368042}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-07 16:56:04,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 88886102836bfd09baf6d56ee561be4c: Running coprocessor pre-open hook at 1675788964658Writing region info on filesystem at 1675788964658Initializing all the Stores at 1675788964661 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788964661Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675788964665 (+4 ms)Cleaning up temporary data from old regions at 1675788964736 (+71 ms)Cleaning up detritus from prior splits at 1675788964736Running coprocessor post-open hooks at 1675788964738 (+2 ms)Region opened successfully at 1675788964738 2023-02-07 16:56:04,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c., pid=150, masterSystemTime=1675788964550 2023-02-07 16:56:04,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:56:04,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-07 16:56:04,739 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:56:04,739 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-07 16:56:04,740 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,740 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,740 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1912): 88886102836bfd09baf6d56ee561be4c/cf2 is initiating minor compaction (all files) 2023-02-07 16:56:04,740 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1912): 88886102836bfd09baf6d56ee561be4c/cf1 is initiating minor compaction (all files) 2023-02-07 16:56:04,740 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 88886102836bfd09baf6d56ee561be4c/cf2 in testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,740 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 88886102836bfd09baf6d56ee561be4c/cf1 in testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,740 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/.tmp, totalSize=34.9 K 2023-02-07 16:56:04,740 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71-bottom] into tmpdir=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/.tmp, totalSize=34.9 K 2023-02-07 16:56:04,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:04,741 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788960600 2023-02-07 16:56:04,741 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675788960600 2023-02-07 16:56:04,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=88886102836bfd09baf6d56ee561be4c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:04,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788964741"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675788964741"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675788964741"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675788964741"}]},"ts":"1675788964741"} 2023-02-07 16:56:04,741 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788960888 2023-02-07 16:56:04,741 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675788960888 2023-02-07 16:56:04,742 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788961642 2023-02-07 16:56:04,742 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675788961642 2023-02-07 16:56:04,742 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788962530 2023-02-07 16:56:04,742 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675788962530 2023-02-07 16:56:04,742 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting 6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788962905 2023-02-07 16:56:04,742 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675788962905 2023-02-07 16:56:04,743 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788963171 2023-02-07 16:56:04,743 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675788963171 2023-02-07 16:56:04,743 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] compactions.Compactor(207): Compacting f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788963424 2023-02-07 16:56:04,743 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] compactions.Compactor(207): Compacting 3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675788963424 2023-02-07 16:56:04,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-07 16:56:04,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure 88886102836bfd09baf6d56ee561be4c, server=jenkins-hbase12.apache.org,40593,1675788846388 in 338 msec 2023-02-07 16:56:04,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-07 16:56:04,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, ASSIGN in 506 msec 2023-02-07 16:56:04,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=850312242221fdade320e947569b2a49, daughterA=88886102836bfd09baf6d56ee561be4c, daughterB=46599e7bdfb807fec0bdc3aed3500139 in 1.3160 sec 2023-02-07 16:56:04,752 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] throttle.PressureAwareThroughputController(145): 88886102836bfd09baf6d56ee561be4c#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:56:04,753 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] throttle.PressureAwareThroughputController(145): 88886102836bfd09baf6d56ee561be4c#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-07 16:56:04,763 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/.tmp/cf1/8113bc03a46f48cbb34fb40fca3be3a5 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/8113bc03a46f48cbb34fb40fca3be3a5 2023-02-07 16:56:04,763 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/.tmp/cf2/71be149159a2426b9f1ef34ad8cea10c as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/71be149159a2426b9f1ef34ad8cea10c 2023-02-07 16:56:04,767 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 88886102836bfd09baf6d56ee561be4c/cf2 of 88886102836bfd09baf6d56ee561be4c into 71be149159a2426b9f1ef34ad8cea10c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:56:04,767 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 88886102836bfd09baf6d56ee561be4c/cf1 of 88886102836bfd09baf6d56ee561be4c into 8113bc03a46f48cbb34fb40fca3be3a5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-07 16:56:04,767 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 88886102836bfd09baf6d56ee561be4c: 2023-02-07 16:56:04,768 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 88886102836bfd09baf6d56ee561be4c: 2023-02-07 16:56:04,768 INFO [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c., storeName=88886102836bfd09baf6d56ee561be4c/cf2, priority=9, startTime=1675788964739; duration=0sec 2023-02-07 16:56:04,768 INFO [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c., storeName=88886102836bfd09baf6d56ee561be4c/cf1, priority=9, startTime=1675788964739; duration=0sec 2023-02-07 16:56:04,768 DEBUG [RS:2;jenkins-hbase12:40593-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:56:04,768 DEBUG [RS:2;jenkins-hbase12:40593-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-07 16:56:04,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40593] ipc.CallRunner(144): callId: 1128 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:36952 deadline: 1675789024931, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49. is not online on jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:05,040 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid 74115c84-26e5-4d15-9a8f-50c13dd5b9f8) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:56:05,040 ERROR [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid a02ccce2-1098-451e-aa52-03b9bc2cf94f) service to localhost.localdomain/127.0.0.1:43315 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-07 16:56:05,051 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:56:05,057 INFO [Listener at localhost.localdomain/34435] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-07 16:56:05,081 DEBUG [Listener at localhost.localdomain/34435] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-07 16:56:05,091 WARN [Listener at localhost.localdomain/34435] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-07 16:56:05,091 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,091 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,092 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,092 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,092 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-07 16:56:05,092 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegion 2023-02-07 16:56:05,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,095 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,095 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788965095"}]},"ts":"1675788965095"} 2023-02-07 16:56:05,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-07 16:56:05,096 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-07 16:56:05,150 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-07 16:56:05,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, UNASSIGN}] 2023-02-07 16:56:05,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, UNASSIGN 2023-02-07 16:56:05,153 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=46599e7bdfb807fec0bdc3aed3500139, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:05,153 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788965153"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788965153"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788965153"}]},"ts":"1675788965153"} 2023-02-07 16:56:05,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 46599e7bdfb807fec0bdc3aed3500139, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:56:05,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, UNASSIGN 2023-02-07 16:56:05,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=88886102836bfd09baf6d56ee561be4c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:05,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788965161"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675788965161"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675788965161"}]},"ts":"1675788965161"} 2023-02-07 16:56:05,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 88886102836bfd09baf6d56ee561be4c, server=jenkins-hbase12.apache.org,40593,1675788846388}] 2023-02-07 16:56:05,196 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,196 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-07 16:56:05,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:05,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 46599e7bdfb807fec0bdc3aed3500139, disabling compactions & flushes 2023-02-07 16:56:05,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:05,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:05,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. after waiting 0 ms 2023-02-07 16:56:05,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:05,312 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71-top] to archive 2023-02-07 16:56:05,314 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:56:05,319 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,323 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,326 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,328 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,330 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,332 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,334 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,335 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794-top, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b-top] to archive 2023-02-07 16:56:05,336 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:56:05,337 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,338 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,339 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,340 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,341 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,342 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,343 DEBUG [StoreCloser-testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:56:05,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:56:05,347 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139. 2023-02-07 16:56:05,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 46599e7bdfb807fec0bdc3aed3500139: Waiting for close lock at 1675788965309Running coprocessor pre-close hooks at 1675788965309Disabling compacts and flushes for region at 1675788965309Disabling writes for close at 1675788965310 (+1 ms)Writing region close event to WAL at 1675788965344 (+34 ms)Running coprocessor post-close hooks at 1675788965347 (+3 ms)Closed at 1675788965347 2023-02-07 16:56:05,348 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:05,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:05,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 88886102836bfd09baf6d56ee561be4c, disabling compactions & flushes 2023-02-07 16:56:05,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:05,349 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=46599e7bdfb807fec0bdc3aed3500139, regionState=CLOSED 2023-02-07 16:56:05,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:05,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. after waiting 0 ms 2023-02-07 16:56:05,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:05,349 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788965349"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788965349"}]},"ts":"1675788965349"} 2023-02-07 16:56:05,351 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71-bottom] to archive 2023-02-07 16:56:05,352 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:56:05,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-07 16:56:05,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 46599e7bdfb807fec0bdc3aed3500139, server=jenkins-hbase12.apache.org,40593,1675788846388 in 196 msec 2023-02-07 16:56:05,354 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/f74fba1ce04f43329336a31cfc713c52.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=46599e7bdfb807fec0bdc3aed3500139, UNASSIGN in 201 msec 2023-02-07 16:56:05,355 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/422a6cff692f4b5e96fd0f69d52e4476.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,356 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/797aa2472df54b1490f1e7c87d783ca3.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,356 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,357 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,358 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/b3a394b523f04e55864aaeb1700863a3.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,358 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/3d3052199798497a8fd6841585fd9e71.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,360 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794-bottom, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49->hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b-bottom] to archive 2023-02-07 16:56:05,360 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-07 16:56:05,361 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/8ca01433cc9e410ab7d23f1e336ee62e.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,362 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/e408284dd6c44d748a9f3c5f64415b11.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,363 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/55b47ccd73ee4bc49904af82068e7e57.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,364 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/451007c8b67a4c9381353241f942109d.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,364 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/6bac32a721024a4094b7df5eef3c59b5.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,365 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f51f933030604c68bfcb465d165ee794.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,366 DEBUG [StoreCloser-testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b.850312242221fdade320e947569b2a49 2023-02-07 16:56:05,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-07 16:56:05,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:56:05,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c. 2023-02-07 16:56:05,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 88886102836bfd09baf6d56ee561be4c: Waiting for close lock at 1675788965349Running coprocessor pre-close hooks at 1675788965349Disabling compacts and flushes for region at 1675788965349Disabling writes for close at 1675788965349Writing region close event to WAL at 1675788965367 (+18 ms)Running coprocessor post-close hooks at 1675788965368 (+1 ms)Closed at 1675788965369 (+1 ms) 2023-02-07 16:56:05,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:05,370 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=88886102836bfd09baf6d56ee561be4c, regionState=CLOSED 2023-02-07 16:56:05,370 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675788965370"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675788965370"}]},"ts":"1675788965370"} 2023-02-07 16:56:05,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-07 16:56:05,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 88886102836bfd09baf6d56ee561be4c, server=jenkins-hbase12.apache.org,40593,1675788846388 in 209 msec 2023-02-07 16:56:05,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-07 16:56:05,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=88886102836bfd09baf6d56ee561be4c, UNASSIGN in 221 msec 2023-02-07 16:56:05,374 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675788965374"}]},"ts":"1675788965374"} 2023-02-07 16:56:05,375 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-07 16:56:05,384 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-07 16:56:05,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 292 msec 2023-02-07 16:56:05,397 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,397 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-07 16:56:05,398 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-07 16:56:05,398 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,399 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegion 2023-02-07 16:56:05,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,403 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,404 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,404 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,404 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-07 16:56:05,411 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 2023-02-07 16:56:05,411 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:05,411 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:05,414 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/recovered.edits] 2023-02-07 16:56:05,414 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/recovered.edits] 2023-02-07 16:56:05,414 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2, FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/recovered.edits] 2023-02-07 16:56:05,418 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/8113bc03a46f48cbb34fb40fca3be3a5 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf1/8113bc03a46f48cbb34fb40fca3be3a5 2023-02-07 16:56:05,418 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/8fee70aa00a045aca0766f9d6d5511cf to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf1/8fee70aa00a045aca0766f9d6d5511cf 2023-02-07 16:56:05,419 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/3d3052199798497a8fd6841585fd9e71 2023-02-07 16:56:05,421 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/422a6cff692f4b5e96fd0f69d52e4476 2023-02-07 16:56:05,421 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/71be149159a2426b9f1ef34ad8cea10c to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/cf2/71be149159a2426b9f1ef34ad8cea10c 2023-02-07 16:56:05,421 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/2c3c03772e704ae6b8966e848ae17711 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/cf2/2c3c03772e704ae6b8966e848ae17711 2023-02-07 16:56:05,422 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/797aa2472df54b1490f1e7c87d783ca3 2023-02-07 16:56:05,423 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/9be176cb44714cfb8f17a3a3b7bb0f6b 2023-02-07 16:56:05,424 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c/recovered.edits/90.seqid 2023-02-07 16:56:05,424 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/recovered.edits/90.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139/recovered.edits/90.seqid 2023-02-07 16:56:05,424 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/88886102836bfd09baf6d56ee561be4c 2023-02-07 16:56:05,424 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/46599e7bdfb807fec0bdc3aed3500139 2023-02-07 16:56:05,425 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/b3a394b523f04e55864aaeb1700863a3 2023-02-07 16:56:05,426 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/d449e5de3a8d4d1c981a1e7b6b5c5178 2023-02-07 16:56:05,427 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf1/f74fba1ce04f43329336a31cfc713c52 2023-02-07 16:56:05,429 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/451007c8b67a4c9381353241f942109d 2023-02-07 16:56:05,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/55b47ccd73ee4bc49904af82068e7e57 2023-02-07 16:56:05,431 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/6bac32a721024a4094b7df5eef3c59b5 2023-02-07 16:56:05,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/8ca01433cc9e410ab7d23f1e336ee62e 2023-02-07 16:56:05,434 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/e408284dd6c44d748a9f3c5f64415b11 2023-02-07 16:56:05,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f50cd6d7e1c4481ebd56bc21bb33d10b 2023-02-07 16:56:05,436 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794 to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/cf2/f51f933030604c68bfcb465d165ee794 2023-02-07 16:56:05,439 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/recovered.edits/85.seqid to hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/archive/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49/recovered.edits/85.seqid 2023-02-07 16:56:05,439 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/.tmp/data/default/testSplitTableRegion/850312242221fdade320e947569b2a49 2023-02-07 16:56:05,439 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-07 16:56:05,441 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,443 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-07 16:56:05,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-07 16:56:05,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-07 16:56:05,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788965445"}]},"ts":"9223372036854775807"} 2023-02-07 16:56:05,446 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788965445"}]},"ts":"9223372036854775807"} 2023-02-07 16:56:05,446 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675788965445"}]},"ts":"9223372036854775807"} 2023-02-07 16:56:05,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-07 16:56:05,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 850312242221fdade320e947569b2a49, NAME => 'testSplitTableRegion,,1675788959596.850312242221fdade320e947569b2a49.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 88886102836bfd09baf6d56ee561be4c, NAME => 'testSplitTableRegion,,1675788963430.88886102836bfd09baf6d56ee561be4c.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 46599e7bdfb807fec0bdc3aed3500139, NAME => 'testSplitTableRegion,41,1675788963430.46599e7bdfb807fec0bdc3aed3500139.', STARTKEY => '41', ENDKEY => ''}] 2023-02-07 16:56:05,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-07 16:56:05,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675788965447"}]},"ts":"9223372036854775807"} 2023-02-07 16:56:05,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-07 16:56:05,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-07 16:56:05,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 59 msec 2023-02-07 16:56:05,505 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-07 16:56:05,506 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-07 16:56:05,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39383] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-07 16:56:05,507 INFO [Listener at localhost.localdomain/34435] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-07 16:56:05,530 INFO [Listener at localhost.localdomain/34435] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375 (was 374) - Thread LEAK? -, OpenFileDescriptor=839 (was 844), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 290), ProcessCount=171 (was 171), AvailableMemoryMB=2775 (was 2918) 2023-02-07 16:56:05,530 INFO [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-07 16:56:05,531 INFO [Listener at localhost.localdomain/34435] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-07 16:56:05,532 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x42379693 to 127.0.0.1:50517 2023-02-07 16:56:05,532 DEBUG [Listener at localhost.localdomain/34435] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,649 DEBUG [Listener at localhost.localdomain/34435] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-07 16:56:05,650 DEBUG [Listener at localhost.localdomain/34435] util.JVMClusterUtil(257): Found active master hash=1633025630, stopped=false 2023-02-07 16:56:05,650 INFO [Listener at localhost.localdomain/34435] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:56:05,679 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 16:56:05,679 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 16:56:05,679 INFO [Listener at localhost.localdomain/34435] procedure2.ProcedureExecutor(629): Stopping 2023-02-07 16:56:05,679 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 16:56:05,679 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-07 16:56:05,679 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:56:05,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:56:05,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:56:05,680 DEBUG [Listener at localhost.localdomain/34435] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d7358eb to 127.0.0.1:50517 2023-02-07 16:56:05,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:56:05,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-07 16:56:05,680 DEBUG [Listener at localhost.localdomain/34435] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,681 INFO [Listener at localhost.localdomain/34435] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,43011,1675788846300' ***** 2023-02-07 16:56:05,681 INFO [Listener at localhost.localdomain/34435] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-07 16:56:05,681 INFO [Listener at localhost.localdomain/34435] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,41937,1675788846347' ***** 2023-02-07 16:56:05,681 INFO [Listener at localhost.localdomain/34435] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-07 16:56:05,681 INFO [Listener at localhost.localdomain/34435] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,40593,1675788846388' ***** 2023-02-07 16:56:05,681 INFO [Listener at localhost.localdomain/34435] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-07 16:56:05,682 INFO [RS:1;jenkins-hbase12:41937] regionserver.HeapMemoryManager(220): Stopping 2023-02-07 16:56:05,682 INFO [RS:2;jenkins-hbase12:40593] regionserver.HeapMemoryManager(220): Stopping 2023-02-07 16:56:05,682 INFO [RS:0;jenkins-hbase12:43011] regionserver.HeapMemoryManager(220): Stopping 2023-02-07 16:56:05,682 INFO [RS:2;jenkins-hbase12:40593] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-07 16:56:05,682 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-07 16:56:05,682 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-07 16:56:05,682 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-07 16:56:05,682 INFO [RS:0;jenkins-hbase12:43011] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-07 16:56:05,682 INFO [RS:2;jenkins-hbase12:40593] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-07 16:56:05,683 INFO [RS:0;jenkins-hbase12:43011] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-07 16:56:05,683 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:05,683 INFO [RS:1;jenkins-hbase12:41937] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-07 16:56:05,683 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(3304): Received CLOSE for 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:56:05,683 DEBUG [RS:2;jenkins-hbase12:40593] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d0d0544 to 127.0.0.1:50517 2023-02-07 16:56:05,683 INFO [RS:1;jenkins-hbase12:41937] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-07 16:56:05,683 DEBUG [RS:2;jenkins-hbase12:40593] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,683 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:05,683 DEBUG [RS:1;jenkins-hbase12:41937] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x30d81752 to 127.0.0.1:50517 2023-02-07 16:56:05,683 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:05,683 INFO [RS:2;jenkins-hbase12:40593] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-07 16:56:05,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 602ba3a55336ac4b5e29ccd78323a197, disabling compactions & flushes 2023-02-07 16:56:05,684 DEBUG [RS:0;jenkins-hbase12:43011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x560f95b6 to 127.0.0.1:50517 2023-02-07 16:56:05,684 DEBUG [RS:1;jenkins-hbase12:41937] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,684 DEBUG [RS:0;jenkins-hbase12:43011] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,684 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,41937,1675788846347; all regions closed. 2023-02-07 16:56:05,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:56:05,684 INFO [RS:2;jenkins-hbase12:40593] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-07 16:56:05,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:56:05,684 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-07 16:56:05,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. after waiting 0 ms 2023-02-07 16:56:05,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:56:05,684 INFO [RS:2;jenkins-hbase12:40593] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-07 16:56:05,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 602ba3a55336ac4b5e29ccd78323a197 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-07 16:56:05,685 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1479): Online Regions={602ba3a55336ac4b5e29ccd78323a197=hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197.} 2023-02-07 16:56:05,685 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-07 16:56:05,685 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-07 16:56:05,686 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-07 16:56:05,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-07 16:56:05,686 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-07 16:56:05,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-07 16:56:05,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-07 16:56:05,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-07 16:56:05,686 DEBUG [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1505): Waiting on 602ba3a55336ac4b5e29ccd78323a197 2023-02-07 16:56:05,686 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-07 16:56:05,686 DEBUG [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-07 16:56:05,699 DEBUG [RS:1;jenkins-hbase12:41937] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs 2023-02-07 16:56:05,699 INFO [RS:1;jenkins-hbase12:41937] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C41937%2C1675788846347:(num 1675788848171) 2023-02-07 16:56:05,699 DEBUG [RS:1;jenkins-hbase12:41937] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,699 INFO [RS:1;jenkins-hbase12:41937] regionserver.LeaseManager(133): Closed leases 2023-02-07 16:56:05,700 INFO [RS:1;jenkins-hbase12:41937] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-07 16:56:05,700 INFO [RS:1;jenkins-hbase12:41937] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-07 16:56:05,700 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 16:56:05,700 INFO [RS:1;jenkins-hbase12:41937] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-07 16:56:05,700 INFO [RS:1;jenkins-hbase12:41937] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-07 16:56:05,702 INFO [RS:1;jenkins-hbase12:41937] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:41937 2023-02-07 16:56:05,715 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/.tmp/info/21766e296cdf41d381fa3c8ed5feb1db 2023-02-07 16:56:05,718 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21766e296cdf41d381fa3c8ed5feb1db 2023-02-07 16:56:05,721 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:05,721 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:05,721 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 16:56:05,721 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,41937,1675788846347 2023-02-07 16:56:05,721 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 16:56:05,721 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 16:56:05,729 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1212873e rejected from java.util.concurrent.ThreadPoolExecutor@162d11a4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,729 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 16:56:05,729 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@649b4e6e rejected from java.util.concurrent.ThreadPoolExecutor@162d11a4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,729 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,41937,1675788846347] 2023-02-07 16:56:05,729 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,41937,1675788846347; numProcessing=1 2023-02-07 16:56:05,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/.tmp/info/25c5a75172e04436b27b6c195c0404d5 2023-02-07 16:56:05,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/.tmp/info/25c5a75172e04436b27b6c195c0404d5 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/info/25c5a75172e04436b27b6c195c0404d5 2023-02-07 16:56:05,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/info/25c5a75172e04436b27b6c195c0404d5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-07 16:56:05,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 602ba3a55336ac4b5e29ccd78323a197 in 57ms, sequenceid=6, compaction requested=false 2023-02-07 16:56:05,749 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,41937,1675788846347 already deleted, retry=false 2023-02-07 16:56:05,749 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,41937,1675788846347 expired; onlineServers=2 2023-02-07 16:56:05,749 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-07 16:56:05,749 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-07 16:56:05,749 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-07 16:56:05,751 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/.tmp/rep_barrier/b4b9ffb3619f4c71ba9fff350ee0a37a 2023-02-07 16:56:05,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/namespace/602ba3a55336ac4b5e29ccd78323a197/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-07 16:56:05,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:56:05,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 602ba3a55336ac4b5e29ccd78323a197: Waiting for close lock at 1675788965684Running coprocessor pre-close hooks at 1675788965684Disabling compacts and flushes for region at 1675788965684Disabling writes for close at 1675788965685 (+1 ms)Obtaining lock to block concurrent updates at 1675788965685Preparing flush snapshotting stores in 602ba3a55336ac4b5e29ccd78323a197 at 1675788965685Finished memstore snapshotting hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675788965685Flushing stores of hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. at 1675788965686 (+1 ms)Flushing 602ba3a55336ac4b5e29ccd78323a197/info: creating writer at 1675788965687 (+1 ms)Flushing 602ba3a55336ac4b5e29ccd78323a197/info: appending metadata at 1675788965700 (+13 ms)Flushing 602ba3a55336ac4b5e29ccd78323a197/info: closing flushed file at 1675788965700Flushing 602ba3a55336ac4b5e29ccd78323a197/info: reopening flushed file at 1675788965737 (+37 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 602ba3a55336ac4b5e29ccd78323a197 in 57ms, sequenceid=6, compaction requested=false at 1675788965742 (+5 ms)Writing region close event to WAL at 1675788965749 (+7 ms)Running coprocessor post-close hooks at 1675788965753 (+4 ms)Closed at 1675788965754 (+1 ms) 2023-02-07 16:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675788848562.602ba3a55336ac4b5e29ccd78323a197. 2023-02-07 16:56:05,755 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b4b9ffb3619f4c71ba9fff350ee0a37a 2023-02-07 16:56:05,771 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/.tmp/table/0a2ece0c0282441588bc909fd8fc93d3 2023-02-07 16:56:05,774 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a2ece0c0282441588bc909fd8fc93d3 2023-02-07 16:56:05,774 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/.tmp/info/21766e296cdf41d381fa3c8ed5feb1db as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/info/21766e296cdf41d381fa3c8ed5feb1db 2023-02-07 16:56:05,777 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21766e296cdf41d381fa3c8ed5feb1db 2023-02-07 16:56:05,778 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/info/21766e296cdf41d381fa3c8ed5feb1db, entries=59, sequenceid=210, filesize=11.1 K 2023-02-07 16:56:05,778 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/.tmp/rep_barrier/b4b9ffb3619f4c71ba9fff350ee0a37a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/rep_barrier/b4b9ffb3619f4c71ba9fff350ee0a37a 2023-02-07 16:56:05,781 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b4b9ffb3619f4c71ba9fff350ee0a37a 2023-02-07 16:56:05,781 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/rep_barrier/b4b9ffb3619f4c71ba9fff350ee0a37a, entries=24, sequenceid=210, filesize=7.5 K 2023-02-07 16:56:05,782 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/.tmp/table/0a2ece0c0282441588bc909fd8fc93d3 as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/table/0a2ece0c0282441588bc909fd8fc93d3 2023-02-07 16:56:05,784 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a2ece0c0282441588bc909fd8fc93d3 2023-02-07 16:56:05,784 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/table/0a2ece0c0282441588bc909fd8fc93d3, entries=36, sequenceid=210, filesize=8.0 K 2023-02-07 16:56:05,785 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 99ms, sequenceid=210, compaction requested=false 2023-02-07 16:56:05,790 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-07 16:56:05,790 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-07 16:56:05,790 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-07 16:56:05,791 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-07 16:56:05,791 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675788965686Running coprocessor pre-close hooks at 1675788965686Disabling compacts and flushes for region at 1675788965686Disabling writes for close at 1675788965686Obtaining lock to block concurrent updates at 1675788965686Preparing flush snapshotting stores in 1588230740 at 1675788965686Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1675788965687 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1675788965689 (+2 ms)Flushing 1588230740/info: creating writer at 1675788965689Flushing 1588230740/info: appending metadata at 1675788965698 (+9 ms)Flushing 1588230740/info: closing flushed file at 1675788965698Flushing 1588230740/rep_barrier: creating writer at 1675788965718 (+20 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675788965730 (+12 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675788965730Flushing 1588230740/table: creating writer at 1675788965755 (+25 ms)Flushing 1588230740/table: appending metadata at 1675788965758 (+3 ms)Flushing 1588230740/table: closing flushed file at 1675788965758Flushing 1588230740/info: reopening flushed file at 1675788965775 (+17 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675788965778 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1675788965782 (+4 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 99ms, sequenceid=210, compaction requested=false at 1675788965785 (+3 ms)Writing region close event to WAL at 1675788965787 (+2 ms)Running coprocessor post-close hooks at 1675788965790 (+3 ms)Closed at 1675788965791 (+1 ms) 2023-02-07 16:56:05,791 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-07 16:56:05,838 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:05,838 INFO [RS:1;jenkins-hbase12:41937] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,41937,1675788846347; zookeeper connection closed. 2023-02-07 16:56:05,838 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1ce88377 rejected from java.util.concurrent.ThreadPoolExecutor@162d11a4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,839 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:41937-0x1013e5847d70002, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:05,839 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4c2edef1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4c2edef1 2023-02-07 16:56:05,839 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@d56a27a rejected from java.util.concurrent.ThreadPoolExecutor@162d11a4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,886 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,43011,1675788846300; all regions closed. 2023-02-07 16:56:05,887 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,40593,1675788846388; all regions closed. 2023-02-07 16:56:05,895 DEBUG [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs 2023-02-07 16:56:05,895 DEBUG [RS:0;jenkins-hbase12:43011] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs 2023-02-07 16:56:05,896 INFO [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C40593%2C1675788846388.meta:.meta(num 1675788848271) 2023-02-07 16:56:05,896 INFO [RS:0;jenkins-hbase12:43011] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C43011%2C1675788846300:(num 1675788848171) 2023-02-07 16:56:05,896 DEBUG [RS:0;jenkins-hbase12:43011] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,896 INFO [RS:0;jenkins-hbase12:43011] regionserver.LeaseManager(133): Closed leases 2023-02-07 16:56:05,896 INFO [RS:0;jenkins-hbase12:43011] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-07 16:56:05,896 INFO [RS:0;jenkins-hbase12:43011] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-07 16:56:05,897 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 16:56:05,897 INFO [RS:0;jenkins-hbase12:43011] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-07 16:56:05,897 INFO [RS:0;jenkins-hbase12:43011] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-07 16:56:05,897 INFO [RS:0;jenkins-hbase12:43011] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:43011 2023-02-07 16:56:05,902 DEBUG [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/oldWALs 2023-02-07 16:56:05,902 INFO [RS:2;jenkins-hbase12:40593] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C40593%2C1675788846388:(num 1675788848171) 2023-02-07 16:56:05,902 DEBUG [RS:2;jenkins-hbase12:40593] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,902 INFO [RS:2;jenkins-hbase12:40593] regionserver.LeaseManager(133): Closed leases 2023-02-07 16:56:05,902 INFO [RS:2;jenkins-hbase12:40593] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-07 16:56:05,902 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 16:56:05,903 INFO [RS:2;jenkins-hbase12:40593] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:40593 2023-02-07 16:56:05,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:05,906 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,43011,1675788846300 2023-02-07 16:56:05,907 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-07 16:56:05,907 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6eceeea8 rejected from java.util.concurrent.ThreadPoolExecutor@6af167a9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:05,917 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,40593,1675788846388 2023-02-07 16:56:05,917 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7344a70 rejected from java.util.concurrent.ThreadPoolExecutor@6af167a9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,917 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@27d5d89c rejected from java.util.concurrent.ThreadPoolExecutor@23ea48be[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:05,927 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,43011,1675788846300] 2023-02-07 16:56:05,928 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,43011,1675788846300; numProcessing=2 2023-02-07 16:56:05,948 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,43011,1675788846300 already deleted, retry=false 2023-02-07 16:56:05,948 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,43011,1675788846300 expired; onlineServers=1 2023-02-07 16:56:05,949 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,40593,1675788846388] 2023-02-07 16:56:05,949 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,40593,1675788846388; numProcessing=3 2023-02-07 16:56:05,959 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,40593,1675788846388 already deleted, retry=false 2023-02-07 16:56:05,959 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,40593,1675788846388 expired; onlineServers=0 2023-02-07 16:56:05,959 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,39383,1675788844945' ***** 2023-02-07 16:56:05,959 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-07 16:56:05,960 DEBUG [M:0;jenkins-hbase12:39383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2068f465, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-07 16:56:05,960 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,39383,1675788844945 2023-02-07 16:56:05,960 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,39383,1675788844945; all regions closed. 2023-02-07 16:56:05,960 DEBUG [M:0;jenkins-hbase12:39383] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-07 16:56:05,960 DEBUG [M:0;jenkins-hbase12:39383] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-07 16:56:05,961 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-07 16:56:05,961 DEBUG [M:0;jenkins-hbase12:39383] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-07 16:56:05,961 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675788847709] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675788847709,5,FailOnTimeoutGroup] 2023-02-07 16:56:05,961 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675788847710] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675788847710,5,FailOnTimeoutGroup] 2023-02-07 16:56:05,961 INFO [M:0;jenkins-hbase12:39383] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-07 16:56:05,961 INFO [M:0;jenkins-hbase12:39383] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-07 16:56:05,961 INFO [M:0;jenkins-hbase12:39383] hbase.ChoreService(369): Chore service for: master/jenkins-hbase12:0 had [] on shutdown 2023-02-07 16:56:05,961 DEBUG [M:0;jenkins-hbase12:39383] master.HMaster(1502): Stopping service threads 2023-02-07 16:56:05,961 INFO [M:0;jenkins-hbase12:39383] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-07 16:56:05,961 ERROR [M:0;jenkins-hbase12:39383] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-07 16:56:05,962 INFO [M:0;jenkins-hbase12:39383] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-07 16:56:05,962 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-07 16:56:05,969 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-07 16:56:05,969 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-07 16:56:05,970 DEBUG [M:0;jenkins-hbase12:39383] zookeeper.ZKUtil(398): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-07 16:56:05,970 WARN [M:0;jenkins-hbase12:39383] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-07 16:56:05,970 INFO [M:0;jenkins-hbase12:39383] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-07 16:56:05,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-07 16:56:05,970 INFO [M:0;jenkins-hbase12:39383] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-07 16:56:05,970 DEBUG [M:0;jenkins-hbase12:39383] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-07 16:56:05,970 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:56:05,970 DEBUG [M:0;jenkins-hbase12:39383] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:56:05,970 DEBUG [M:0;jenkins-hbase12:39383] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-07 16:56:05,970 DEBUG [M:0;jenkins-hbase12:39383] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:56:05,970 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.59 KB heapSize=719.62 KB 2023-02-07 16:56:06,028 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:06,028 INFO [RS:0;jenkins-hbase12:43011] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,43011,1675788846300; zookeeper connection closed. 2023-02-07 16:56:06,028 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7fbbbfc4 rejected from java.util.concurrent.ThreadPoolExecutor@6af167a9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:06,028 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:43011-0x1013e5847d70001, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:06,028 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@72fb6d69] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@72fb6d69 2023-02-07 16:56:06,028 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a9eaf9d rejected from java.util.concurrent.ThreadPoolExecutor@6af167a9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:06,128 INFO [RS:2;jenkins-hbase12:40593] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,40593,1675788846388; zookeeper connection closed. 2023-02-07 16:56:06,128 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:06,128 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5e5c91e7 rejected from java.util.concurrent.ThreadPoolExecutor@23ea48be[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:06,128 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4b732d0a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4b732d0a 2023-02-07 16:56:06,128 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): regionserver:40593-0x1013e5847d70003, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:06,129 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@261aae66 rejected from java.util.concurrent.ThreadPoolExecutor@23ea48be[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:06,129 INFO [Listener at localhost.localdomain/34435] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-07 16:56:06,384 INFO [M:0;jenkins-hbase12:39383] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.59 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f8ca17f1c596422cb929ed4b3df2406a 2023-02-07 16:56:06,393 INFO [M:0;jenkins-hbase12:39383] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f8ca17f1c596422cb929ed4b3df2406a 2023-02-07 16:56:06,394 DEBUG [M:0;jenkins-hbase12:39383] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f8ca17f1c596422cb929ed4b3df2406a as hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f8ca17f1c596422cb929ed4b3df2406a 2023-02-07 16:56:06,396 INFO [M:0;jenkins-hbase12:39383] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f8ca17f1c596422cb929ed4b3df2406a 2023-02-07 16:56:06,396 INFO [M:0;jenkins-hbase12:39383] regionserver.HStore(1080): Added hdfs://localhost.localdomain:43315/user/jenkins/test-data/b1246a76-c0a6-d8f8-9c9f-077b06e135af/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f8ca17f1c596422cb929ed4b3df2406a, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-07 16:56:06,397 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegion(2947): Finished flush of dataSize ~608.59 KB/623192, heapSize ~719.60 KB/736872, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 427ms, sequenceid=1282, compaction requested=false 2023-02-07 16:56:06,398 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-07 16:56:06,398 DEBUG [M:0;jenkins-hbase12:39383] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675788965970Disabling compacts and flushes for region at 1675788965970Disabling writes for close at 1675788965970Obtaining lock to block concurrent updates at 1675788965971 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675788965971Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623192, getHeapSize=736872, getOffHeapSize=0, getCellsCount=1499 at 1675788965971Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675788965971Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675788965971Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675788965977 (+6 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675788965977Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675788966394 (+417 ms)Finished flush of dataSize ~608.59 KB/623192, heapSize ~719.60 KB/736872, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 427ms, sequenceid=1282, compaction requested=false at 1675788966397 (+3 ms)Writing region close event to WAL at 1675788966398 (+1 ms)Closed at 1675788966398 2023-02-07 16:56:06,400 INFO [M:0;jenkins-hbase12:39383] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-07 16:56:06,400 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-07 16:56:06,401 INFO [M:0;jenkins-hbase12:39383] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:39383 2023-02-07 16:56:06,422 DEBUG [M:0;jenkins-hbase12:39383] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase12.apache.org,39383,1675788844945 already deleted, retry=false 2023-02-07 16:56:06,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:06,533 INFO [M:0;jenkins-hbase12:39383] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,39383,1675788844945; zookeeper connection closed. 2023-02-07 16:56:06,533 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2cc68aef rejected from java.util.concurrent.ThreadPoolExecutor@6b3103b0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:06,533 DEBUG [Listener at localhost.localdomain/34435-EventThread] zookeeper.ZKWatcher(600): master:39383-0x1013e5847d70000, quorum=127.0.0.1:50517, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-07 16:56:06,533 ERROR [Listener at localhost.localdomain/34435-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@bf31428 rejected from java.util.concurrent.ThreadPoolExecutor@6b3103b0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-07 16:56:06,535 WARN [Listener at localhost.localdomain/34435] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-07 16:56:06,540 INFO [Listener at localhost.localdomain/34435] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-07 16:56:06,658 WARN [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-07 16:56:06,658 WARN [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid 3f3daf44-903e-4489-b9da-57bdf6b56058) service to localhost.localdomain/127.0.0.1:43315 2023-02-07 16:56:06,661 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data5/current/BP-275525888-136.243.104.168-1675788840051] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 16:56:06,661 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data6/current/BP-275525888-136.243.104.168-1675788840051] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 16:56:06,663 WARN [Listener at localhost.localdomain/34435] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-07 16:56:06,670 INFO [Listener at localhost.localdomain/34435] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-07 16:56:06,775 WARN [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid 74115c84-26e5-4d15-9a8f-50c13dd5b9f8) service to localhost.localdomain/127.0.0.1:43315 2023-02-07 16:56:06,778 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data3/current/BP-275525888-136.243.104.168-1675788840051] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 16:56:06,779 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data4/current/BP-275525888-136.243.104.168-1675788840051] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 16:56:06,781 WARN [Listener at localhost.localdomain/34435] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-07 16:56:06,783 INFO [Listener at localhost.localdomain/34435] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-07 16:56:06,887 WARN [BP-275525888-136.243.104.168-1675788840051 heartbeating to localhost.localdomain/127.0.0.1:43315] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-275525888-136.243.104.168-1675788840051 (Datanode Uuid a02ccce2-1098-451e-aa52-03b9bc2cf94f) service to localhost.localdomain/127.0.0.1:43315 2023-02-07 16:56:06,888 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data1/current/BP-275525888-136.243.104.168-1675788840051] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 16:56:06,888 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/47c88641-eb07-ff2b-f399-0ac9f01ce9fa/cluster_6fefafd9-e61a-0117-f17c-5f999fd5d603/dfs/data/data2/current/BP-275525888-136.243.104.168-1675788840051] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-07 16:56:06,910 INFO [Listener at localhost.localdomain/34435] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-07 16:56:07,092 INFO [Listener at localhost.localdomain/34435] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-07 16:56:07,317 INFO [Listener at localhost.localdomain/34435] hbase.HBaseTestingUtility(1293): Minicluster is down